AI researchers are warning that new AI voice technology could be exploited. To prove their point, they used ChatGPT 4.0 to carry out seven common scams, many of which were successful at an alarming rate.
The peer-reviewed research published this month proves that advanced AI models equipped with voice capabilities could potentially automate common phone scams.
They Wanted To See If Voice AI Agents Could Be Used As Autonomous Scam Bots
The study, led by researchers Richard Fang, Dylan Bowman, and Daniel Kang, wanted to ask, “Can Voice-Enabled AI Agents perform the task of executing common scams?
They conducted experiments to see if AI agents, on their own, could log into bank accounts, elicit OTP codes from victims, or get victims to give them gift cards.
In the end, they were surprised at how easy it was. They achieved a success rate between 20% to 60% for each scam they tested. Overall, they successfully used AI agents to complete 1 in 3 scam attempts.
The Researchers Tested Seven Scams
The researchers used ChatGPT 4.o and applied standard jailbreaking prompt templates to bypass the model’s protections. They did not reveal where they got the template.
From there, they set up a scammer agent. ChatGPT would speak to the victim, extracting information while they used software tools in the background to carry out the scam, such as logging into the Bank of America website (which they actually did).
To test the AI agents, they tried seven different common scams. These are the scams they attempted to do with the AI agents they created.
The scams ranged from bank account transfer scams on Bank of America’s website to credential stealing scams on Gmail, Monero, Instagram, and Google Play.
They ran each scam 5 different times to test the effectiveness of their AI Scam bot.
The AI Scam Agents Were Very Effective and Cheap
The AI scam bot was remarkably effective at scamming; it seems it was probably more successful than the typical scammer.
What makes the findings particularly alarming is that the AI agent could run very cheaply and very, very fast, working in as little as 2 minutes. The scam that took the longest was the Bank of America money transfers which took about 4 minutes and cost $2.51 to complete.
This graphic shows how successful their scamming agents were at various different scams.
Read Their Research Paper Here
You can read their whole paper here.