Do phishing simulations work? Sometimes

Phishing simulations are becoming increasingly popular to pinpoint which employees fall victim to scams, but their effectiveness and morality have been called into question.

Eighty-three percent of organizations reported experiencing a successful email-based phishing attack in 2021, with 54% responding they dealt with more than three successful attacks in the same year, according to a Proofpoint survey.

Phishing attacks, which were up 26% in 2021 over 2020, are one of the leading causes of data breaches. Preventing employees, partners and contractors from falling victim to these scams is paramount.

Social engineering penetration testing, which often includes sending mock phishing emails to employees to gauge their security awareness, has become a common practice to combat the threat. But these tests are under hot debate, with many questioning their efficacy, as well as if the negatives -- such as creating distrust and damaging company morale -- outweigh the benefits.

Do phishing simulations work?

Cybersecurity awareness trainings have remained relatively stagnant over the past decade, according to Jinan Budge, analyst at Forrester Research. The widespread adoption of phishing simulations has been the only major development in recent years, she added.

Phishing tests are used to collect data on employees' click and response rates to malicious emails, links and attachments. The simulations are meant to help employees recognize malicious emails.

But do they have any lasting effect?

A 2021 study from ETH Zurich, a public research university in Switzerland, concluded phishing simulations -- at least embedded phishing tests, which tell employees when they clicked a phishing link or send an employee who clicked a malicious email to voluntary training -- don't necessarily help reduce click and dangerous action rates among employees.

ETH Zurich researchers concluded that "embedded phishing training is … not effective and can in fact have negative side effects," suggesting tests make users more susceptible to attacks because employees either gain false confidence from the trainings or start to feel less responsible for stopping such attacks.

The researchers did find crowdsourced phishing simulations, on the other hand, to be effective. Participants in the experiment were given a button to alert the security team of a suspicious email. They correctly reported 68% of phishing emails, aiding the security team in phishing prevention.

The study did say, however, that even crowdsourced phishing simulations should complement other security awareness measures.

Gartner analyst William Candrick agreed. Phishing simulations alone aren't effective, he said, unless an organization has a program to engage repeat offenders.

Enterprises should use data from phishing tests to identify high-risk employees and then enforce mandatory and interactive cybersecurity awareness trainings.

Are phishing simulations ethical?

Despite their ability to identify risky employees, phishing simulations remain controversial, even among security professionals.

In September 2020, Tribune Publishing Company sent employees an email offering bonuses of up to $10,000. After clicking the link to see how much their bonus would be, however, employees learned they failed a phishing test -- and would receive no bonus. Employee backlash, viral social media posts and bad publicity ensued, especially as the email was sent out after several recent years of layoffs and furloughs -- not to mention amid a global pandemic.

But that doesn't mean the tests are wrong, per se.

"Everyone gets mad [about phishing simulations] … but the IT department is perfectly in the right, and the cybersecurity team is doing exactly what it should be doing," said Johna Till Johnson, CEO of Nemertes Research. "Hackers are not sensitive to the feelings of employees."

While it may be legal, it isn't always moral. "You have to consider many factors when conducting a simulation," Budge said. She suggested asking the following phishing simulation questions before deploying a test:

  • How will it impact employees' mental health?
  • Is the simulation necessary?
  • How will the message be perceived?
  • Does this benefit employees?
  • Are we being smug, or do we genuinely want to change behaviors?
  • Is there a better way to communicate this message?

Asking these questions also helps prevent security teams from ostracizing themselves from the rest of the organization, she added.

It's also important for security teams to teach and regularly reiterate that cybersecurity is a team sport. Security is often viewed as IT's problem; 70% of employees believe it is IT's responsibility to ensure company accounts are not breached or hacked. Without the proper messaging, phishing simulations only reinforce this mindset.

If a company does use phishing simulations, Candrick suggested security teams conduct extended phishing campaigns with increased difficulty over time. This helps employees who are doing well continue to improve and feel rewarded for their efforts.

Phishing is here to stay -- and so are phishing simulations, at least for the time being. While evidence shows phishing simulations alone are ineffective, they can be useful in conjunction with risk management practices and a security awareness program that uses the data collected from these tests to identity and train high-risk employees.

"Phishing simulations are a necessary evil at the moment," Budge said.

Next Steps

How to prepare for ransomware and phishing attacks

3 types of phishing attacks and how to prevent them

How text messaging phishing can affect the enterprise

Dig Deeper on Security operations and management

Enterprise Desktop
  • Understanding how GPOs and Intune interact

    Group Policy and Microsoft Intune are both mature device management technologies with enterprise use cases. IT should know how to...

  • Comparing MSI vs. MSIX

    While MSI was the preferred method for distributing enterprise applications for decades, the MSIX format promises to improve upon...

  • How to install MSIX and msixbundle

    IT admins should know that one of the simplest ways to deploy Windows applications across a fleet of managed desktops is with an ...

Cloud Computing