Real-world AI voice cloning attack: A red teaming case study
Thanks to AI, social engineering campaigns are becoming more effective than ever. In this red team case study, see how voice cloning helped trick a seasoned business user.
As an ethical hacker, I put organizations' cyberdefenses to the test, and -- like malicious threat actors -- I know that social engineering remains one of the most effective methods for gaining unauthorized access to private IT environments.
The Scattered Spider hacking group has repeatedly proven this point in its social engineering attacks targeting IT help desks at major enterprises, including casino giants Caesars Entertainment and MGM Resorts, as well as British retailer Marks and Spencer. In such attacks, a threat actor impersonates a legitimate employee and convinces the help desk to reset that user's password, often using an authoritative tone or sense of urgency to manipulate the other person into granting account access. Such classic social engineering tactics often manage to bypass technical defenses entirely by exploiting human behavioral weaknesses.
I've used phone-based social engineering in my own red teaming strategy for years, and recent improvements in deepfake and voice cloning technology have made such voice phishing (vishing) attacks even more effective. In this article, I will walk you through a recent, real-world example that demonstrates how easily threat actors are now using AI-enabled deepfakes and voice cloning to deceive end users. CISOs must test their organizations' ability to withstand such attacks, as well as educate employees on what these techniques look like and how to stop them.
How an AI voice cloning attack tricked a seasoned employee
As part of a red teaming exercise, a large business recently asked me to try to hack into the email account of one of its senior leaders. Typically, you need the following three elements to gain access to an email account:
- The email address.
- The password.
- A method of bypassing MFA.
In this case, the target's email address itself was listed publicly. His information had also been exposed in several public data breaches, with the same password apparently in use across multiple separate accounts. I surmised he was likely to use the same password for his corporate account login, as well.
Defeating the company's MFA, Microsoft Authenticator, was the trickiest part of the red team exercise. I decided the best method would be to call the target and impersonate a member of the company's IT team, using voice cloning.
First, I identified the names of the organization's IT team members on LinkedIn and then further researched them on Google. I found that one of the senior IT leaders had given a presentation at a conference, with a 60-minute video of the session publicly available on YouTube. It is possible to clone someone's voice with just three seconds of audio, so I was confident an hour-long recording would enable a very accurate and convincing replica.
I extracted the audio from the YouTube video and used a tool called ElevenLabs to create a voice clone. I then attempted to log in to the target's email account using the password I had found exposed in previous third-party data breaches, and as anticipated, it worked.
The successful login triggered Microsoft Authenticator, sending the target an MFA push notification on his phone. I called him, using the AI voice cloning software to impersonate the IT team member in our real-time conversation. I explained to the target that the IT team was conducting internal maintenance on his account, leading to the MFA prompt, and asked him to enter the two-digit number from my screen into his Microsoft Authenticator app. Completely convinced, he typed in the number, thereby giving me access to his email and SharePoint.
The target had been with the company for 15 years at the time of the red team exercise, so his account held a treasure trove of information. If I had been a malicious hacker, I could have started sending email from his real email address, potentially tricking further staff members or clients into opening malicious documents or authorizing financial transactions.
Lessons learned
This example demonstrates why I have been unsurprised to see criminal groups increasingly turning to vishing-based social engineering as a reliable method for gaining initial access to target environments. Once a threat actor has accessed a Microsoft business account -- especially one with elevated privileges -- compromising the network and running ransomware on all endpoints and important servers is relatively simple.
To protect against these types of attacks, CISOs must ensure IT support teams follow clear and consistent verification procedures in conversations with end users. Most importantly, organization-wide security awareness training should educate all employees about these types of attacks, the psychological tricks they employ and best practices for verifying that someone is who they claim to be.
Rob Shapland is an ethical hacker specializing in cloud security, social engineering and delivering cybersecurity training to companies worldwide.
