AI models designed to closely simulate the human voice make it easier for bad actors to imitate loved ones or trick vulnerable people out of thousands of dollars. Yes, reports The Washington Post.
Some rapidly sophisticated AI speech generation software requires just a few sentences of speech to produce compelling speech that conveys the tone and emotional tone of the speaker’s voice. For example, some require only 3 seconds. Elderly people are often targeted, according to a Washington Post report, and it’s harder to detect that a voice isn’t real, even when the exigencies described by scammers seem implausible. can become increasingly difficult.
Advances in technology have made it easier to prey on people’s worst fears, and he said he felt an “instinctive dread” when he heard what sounded like direct pleas from friends and family in need. Scaring victims who told the Post. A couple transferred $15,000 to scammers via a bitcoin terminal, believing they had spoken to their son. An AI-generated voice said he needed legal fees after being involved in a car accident that killed a US diplomat.
So-called identity theft scams are very common in the United States, according to the Federal Trade Commission. This was the most frequently reported type of fraud reported in 2022 and resulted in her second highest loss to those targeted. Out of 36,000 reports, over 5,000 victims phoned him out of $11 million.
The Post reports that these impersonation scams can be carried out from anywhere in the world, making it very difficult for authorities to crack down on them and reverse the alarming trend. Not only is it difficult to track calls, identify scammers, and recover funds, but it is also difficult to determine which agency has jurisdiction to investigate individual cases when scammers operate in different countries. there is. Even if it is clear which agencies should investigate, some agencies are currently ill-equipped to deal with the growing number of impersonations.
Ars was unable to reach the FTC for comment. Will Maxson, assistant director of the FTC’s marketing practice, told the Post that raising awareness of fraud using AI voice simulators is likely the best defense for consumers today. increase. He recommends treating requests for cash with skepticism. Before you send money, try contacting people you think need help in ways other than voice calls.
Protection against AI voice spoofing
AI voice modeling tools are being used to improve text-to-speech generation, create new possibilities for voice editing, and extend the magic of cinema by replicating famous voices like Darth Vader. But its ability to easily create convincing voice simulations has already caused a scandal. No one knows who should be held responsible if this technology is misused.
Earlier this year, there was backlash when some 4chan members created deepfake voices of celebrities making racist, offensive, or violent remarks. At that point, it became clear that companies should consider adding more safety measures to prevent the misuse of technology. We have reported that we may be held liable for causing damage.
The court has decided when companies will be held liable for damages caused by deepfake voice technology, which appears to pose an increased risk of defamation and misinformation, as well as other popular AI technologies such as ChatGPT. Haven’t decided yet.
However, many companies appear to be releasing AI products without a full understanding of the risks associated with them, potentially increasing pressure on courts and regulators to put AI in check. .
Now, some companies seem reluctant to delay the release of popular AI features, including the controversial one that allows users to emulate celebrity voices. Most recently, during a preview of his Bing AI, Microsoft rolled out a new feature that can be used to emulate celebrities, reports Gizmodo. With this feature, Microsoft seems to be trying to avoid any kind of scandal by limiting celebrity voice impersonation.
Microsoft did not respond to Ars’ request for comment on how safeguards are currently in place to prevent celebrity voice emulators from generating offensive remarks. Gizmodo points out that Microsoft, like many companies looking to profit from the widespread appeal of AI tools, is counting on its millions of users to beta test “still-dysfunctional AI.” bottom. as a parody. Only time will tell how effective early solutions are in mitigating risk.
In 2021, the FTC will release AI guidance telling businesses that products should “do more good than harm” and that businesses should be prepared to take responsibility for the risks of using their products. I was. Most recently, the FTC last month told companies that they “need to know the reasonably foreseeable risks and impacts of AI products before they hit the market.”
