Enterprises often talk about the need for cyber-war gaming but struggle when it comes to the nuts and bolts of conducting one. Information is available on how to construct and implement a cyber-war game, but putting one into practice is another question.
The following case study is an account of the experiences of a real-world CISO conducting a cyber-war game. The company is a midsize firm of around 3,000 employees with distributed operations across the U.S. It has a heavy emphasis on knowledge work and no manufacturing plants or retail stores.
Editor's note: The company that conducted the cyber-war game wishes to remain anonymous. The author has annotated the interview with the CISO with observations and insights.
What inspired the decision to cyber-war game?
"We were really being pushed by our board to do better on ransomware. We've never been hit, but we realize the risk is much greater than it used to be. So, we focused on what do we need to do make ourselves more prepared and more resilient," the CISO said.
In Nemertes' experience, this is typical. The decision to hold cyber-war games comes from senior-level management, including the CEO and, sometimes, the board. The concern is that executives and the board lack insight into what might happen in the event of an attack, and they want something more concrete than assurances from the CISO.
How did you prepare for the cyber-war game?
In preparation for the war game, the CISO said, the security team completely revised and updated the company's incident response plan (IRP). It then developed a ransomware-specific version of the IRP.
After those two tasks were completed, the CISO said the team conducted "a technical incident response focused on detection, containment and restoration of services."
In Nemertes' view, this is the right way to prepare for a cyber-war game. The company's board was specifically concerned about ransomware, so the security team made sure it had both a general-purpose IRP and a ransomware-specific playbook.
Before conducting any kind of cyber-war game or tabletop exercise, companies should have, at minimum, preliminary guidelines on how to respond.
Did you use a partner? If so, how did you select a partner?
The CISO reported that, as with the vast majority of CISOs Nemertes works with, the company worked with an outside firm.
When selecting the partner, the CISO looked at the following things:
- its expertise and capabilities;
- how it viewed a ransomware cyber-war game as different from other cyber-war games; and
- its ongoing history and relationship with the firm.
Because he and his company had a positive experience with the provider previously, the CISO said it made the decision to repeat the exercise with that partner.
What outputs were you looking to achieve?
The main objectives, the CISO said, were to get everybody aware of how ransomware attacks are different and for both the security team and the organization as a whole to learn how a ransomware-specific playbook works. "There [were] a lot of new people who hadn't previously gone through the tabletop exercise," he added.
In general, "awareness of gaps in the response" is one of the main goals most organizations have for cyber-war gaming. By going through the exercise, security teams can flag areas for improvement.
Were those results achieved?
The CISO said the cyber-war game exercise delivered the intended results. The primary gap the company found was -- not surprisingly -- in the area of communication. "Even though we emphasized improving our communications aspect, there still are communication gaps," he said.
In this company's case, however, the issue had more to do with how to communicate effectively than knowing who to communicate with or how to reach them. In an unrelated episode, there was an urgent need to reach IT and cybersecurity staff. The CISO found that emailing employees after hours didn't work; they weren't reading emails. But the leaders who called or texted their teams were able to reach them right away.
The lesson? "In an incident, you have to be calling people," the CISO said.
In Nemertes' experience, all of this is typical. Communications is almost always the greatest gap in any IRP. Knowing how to communicate effectively with different individuals creates a real challenge in today's multichannel world. Some people -- like this CISO's team -- read texts but not email. Others respond to phone calls but not text or email, while still others may respond primarily via enterprise collaboration tools, such as Slack or Teams.
The bottom line? To be effective, an IRP must specify not just who to reach but how to reach them. The channel also needs to match the proclivities of the team. Don't try to tell a text-centric team it needs to take phone calls or answer email.
Was this exercise in person or virtual?
"It was done virtually. But the fact is: We're a virtual organization," the CISO said, noting that the virtual exercise was about as effective as its previous in-person one before the COVID-19 pandemic.
This aligns with Nemertes' experience. We conduct cyber-war games virtually, and our clients report a high degree of satisfaction with the outcome.
Who was involved in the exercise? Did it extend beyond the technical team?
This particular exercise involved only the technical team, the CISO said, but the company is planning a broader exercise later in the year that includes management.
In Nemertes' experience, this is the right strategy. If an organization has never done a cyber-war game before, do the first one with the technical team. Once confident all obvious gaps have been identified and addressed, extend the exercise to the broader organization.
What would you do differently in the future?
Even a successful exercise can be improved upon, and this one was no exception.
"A tabletop exercise requires a good deal of planning and coordination," the CISO said. "There are a variety of scenarios. There should have been more planning on different kinds of scenarios to stress different components of the IRP."
This is a key point. Focus cyber-war gaming exercises on areas that require emphasis. If communication is a weak link in your organization, for example, make sure the cyber-war game addresses communication. Or, if the tooling or automation to successfully contain a type of breach is lacking, make sure the cyber-war game includes that specific type of breach.
What additional advice do you have?
"Tabletop exercises really are valuable. We don't do them often enough," the CISO said. "We can send documents; people can read things. But it's not until everyone's virtually in the room where you really learn what everybody's responsibility is. It's like any other war gaming -- the best learning happens when you're trying to practice what happens in real life."
Nemertes concurs. We recommend cyber-war gaming at least twice per year, which is this CISO's recommendation as well. That said, quarterly cyber-war gaming is optimal, but the CISO feels that might be too much overhead.
The bottom line? Cyber-war gaming is an effective tool in any threat mitigation portfolio. If your organization has never held one before, there's no time like the present. And, if you have tried cyber-war gaming in the past, it's time to take it to the next level.