Russia-Ukraine war prompts security best practices refresher
The Russia-Ukraine war hasn't led to major cyber attacks affecting MSPs, but IT services providers should review cybersecurity best practices, including hardening and response.
Dave Sobel is host of the podcast The Business of Tech and co-host of the podcast Killing IT. In addition, he wrote Virtualization: Defined. Sobel is regarded as a leading expert in the delivery of technology services, with broad experience in both technology and business.
In this video, Sobel looks at what the Russia-Ukraine war means for cybersecurity in the IT services space with Blackpoint Cyber CEO Jon Murchison. Murchison discusses a worst-case scenario and what best practices IT services providers should have in place to protect their business and customers.
Transcript follows below. Minor edits have been made for brevity and clarity.
Dave Sobel: I wanted to bring somebody in from the security space to talk to me a little bit about the current situation related to Ukraine and Russia. It's on all of our minds. And, from a cyber perspective, I think most of us in IT services are looking, going like, 'Well, it feels like attacks are going a little lower because they're focused on one another.' What are the experts seeing? What are you seeing?
Jon Murchison: This is a really big topic. As soon as this popped off, it was like instant security theater from every marketing department, like 'The world's going to come down.' We internally recorded a private video, not for public consumption, of our thoughts to our customer base. And here's the uncut, just pragmatic take. We have seen zero increase in volume or sophistication of attacks, either from nation-states or ransomware groups. Now, when we talk to the MSP space, I would say ransomware is 95% of what we're stopping on pretty much a daily basis. Nation-states are more like 2%, just because [for] the typical customer an MSP serves -- unless it's like a regional utility or some small boutique manufacturer in a critical supply chain for a defense or aircraft or whatever -- you just don't see [nation-states attack] a lot.
With that being said, it is plausible to think that maybe the Russian-based ransomware groups could feel more empowered to increase their operations. At the same time, Conti had some issues. Conti is obviously one of the biggest Russian-based ransomware syndicates. And, like all these syndicates, they're folks from all different countries. I don't know if they had Ukrainian guys in the game or not, but they probably had some pretty massive leaks that has caused for us to see different tradecraft.
Big picture, it's a good reminder to button up on the fundamentals. When nation-states go to war, you must assume the nation-state assets coalesce to focus on that topic. And they can't take on everyone at once. They have a lot of things going on. They're getting bogged down, obviously, in the conventional military side. They have vigilante cyber groups going after them. That has to eat up some of their defensive capabilities. And I would think their offensive guys are still primarily focused on Ukraine. That's not to say there isn't a threat here in the United States, that we shouldn't be prepared. We should. But I think you're really looking more at banking, big manufacturers, logistics. Then, critical infrastructure -- I don't know where that magic red line is because this has never really happened, like full-scale nation-state-on-nation-state warfare. I have to think there's a red line, but I have to think the bar would be a little bit lower for regional municipalities. We see a lot of combined networks. Big picture, we haven't seen anything yet, but we're prepared if we do.
Sobel: Is that the extent of the collateral damage? Do you think there's collateral damage we need to be thinking about in this conflict?
Murchison: I think the collateral damage will happen if offensive actions are taken against Western countries, where it's like, 'This is clearly a nation-state, not just like a criminal syndicate or something like that.' I haven't seen it happen, or I don't know enough deep-in-the-weeds technical details, [but] I have read that there have been -- and I have no inside info -- that there have been some attacks on Ukrainian ISP internet infrastructure. That has always been my greatest concern. Utilities and SCADA [supervisory control and data acquisition] attacks and all this stuff, and banks get all the press. But the reality is SCADA attacks are still pretty hard to pull off. And, when you look at the one the Russians did against the Ukrainians a bunch of years ago, did it shock everyone? Yes. It took the power out for about 230,000 people, for like [one to six hours], but there's actually no SCADA protocol or [programmable logic controller] attack in that. It was actually more 'live off the land, literally open the breaker' type attack and destroy everything on the way out. And, frankly, we have thunderstorms in the spring and summer that do way worse here all the time. Those are really hard. I worry much more about routing and switching infrastructure at the core. That is the glue that everyone sleeps on that is, by far, in my opinion, the most critical infrastructure when it comes to internet [communications] because it just has cascading effects everywhere.
Sobel: I'm going to ask you, then, to take out your crystal ball, and the typical caveats on this, it's a crystal ball. It's cracked, and it's foggy.
Murchison: So, I'm just going to make it up.
Sobel: Right. So, we know it's not great. Talk to me about how you think this might play out, and maybe we'll do sort of two scenarios to give you a little sense. What's the worst way this might play out, cyber-wise, and what we'd be hoping for on the good end of the spectrum?
Murchison: I think the worst way this plays out on the cyber side is there's large-scale attacks against internet communications, infrastructure, media infrastructure, banking, manufacturing and logistics. That being said … I mean, I come from the intelligence community world. I also think that's really hard to pull off and time it right. These things just don't work. There's not physics involved quite the same as launching a missile and it hits the target. You fail way more than you ever win. Achieving effect with timing, achieving the ultimate mission goal, it's quite difficult, and it requires a lot of things to go your way. And I also can't imagine doing it while being attacked at the same time. It would actually be quite difficult. I think, on the worst-case scenario, they just decide to go all in on cyber warfare. I got to be honest, I think, if that happens, this is escalating to a whole other ballgame. We need to be candid, cyber is like the least of our worries. I worry much, much more [about other types of attacks], to be candid, and way less about the cyber side. I worry way more on this going something past conventional or something where our satellite infrastructure is.
Sobel: I want to do a double-check. I think the answer is the standard best practices, but I want to double-check it. From a typical IT services company, serving a collection of customers, are the recommendations essentially just continue good best practices, or is there something unique to this situation?
Murchison: No, I think there's more, but I don't think it's unique to Russia-Ukraine. I think it's unique to the threat of an attack on your continuity of operations, whether it's ransomware, or data theft, or the holding you hostage, or a destructive attack, like the wiper stuff we see. Industry best practices mostly, to me, focus on reducing your attack surface. The way we like to look at it -- and it's kind of a riff of a [NIST] model, called the Cyber Defense Matrix that Sounil Yu made -- but we take a step back, and you look at any infrastructure, and this is taking compliance out of it because I consider that mostly checkbox security. In bucket one, you need to have some capability to identify your assets' patch level, what you have. And that's actually something we see people whiff on all the time. You have to know what you need to protect and have some live visibility into it. That's step one.
Step two, which is where I see a lot of the best practices come in, is hardening. This is IT hygiene. This is everything from locking down your remote monitoring and management, multifactor authentication, regular patching, continuous external vulnerability scanning specifically that reduces your susceptibility to remote code execution. That's a key area.
And then, the next two areas [are] where I diverge with industry best practices, kind of in air quotes, because, when you look at detection, most cyber malicious activity breach detection is so myopically focused on the malicious tool set. And the dirty secret, if you've ever done this game before, is your malicious tool sets are part of the game, but so much of what you do on the keys looks like clever system administration. You're trying to figure out where you are, your testing cred, your port scanning, and that's what we call 'tradecrafter behavior.' And that's where these two worlds need to be merged together. And, frankly, there just aren't a lot of people that are good at it out there still -- because so much of the dev effort goes towards catching the malicious software. I always say, if you ask a software engineer -- which I consider the biggest brain guys -- to catch a hacker, they're going to try and catch what they would make if they were on the offensive side, which is persistence techniques, exploit techniques and then just malware in general. If you ask the guy in the keys, many times, he's going to be looking for some amount of dollar sign chair, all that live off the land, privileged creds, lateral spread. I think it's really important in the tech, you merge those two areas.
And then the third part is response. This is where our industry has kind of bastardized the term a lot. But what it really should mean is real-time response. If you don't have a capability at two in the morning to operate within, assume your breach and assume your automated security tools don't succeed, how are you going to take action? And I think that's where the industry's changing. The typical model was, I get a SIEM [security information and event management], I gobble up terabytes of logs, I make dashboards and I have threat hunting. Well, that does not work at line rate, period. It just does not work fast enough, but it's great for compliance. It's great for checking the boxes. It's great for forensics post-breach.
And then the last bit would be really the right of boom, assuming all went wrong, and that's best practices, cloud backups, having a disaster recovery plan, just making sure you have your insurance and [incident response] folks, that you actually know who to call. I think part of it is what capabilities you have in place, combined with best practices on hardening.
Sobel: I'm going to ask you a really targeted one around best practices because it's one that I feel gets an interesting debate. The head of the U.K. cyber center actually came out recently and sort of commented, 'I think most businesses should go to [automated] patching because patching is a pretty standardized thing these days. It's pretty well maintained. These big companies generally know what they're doing. And, by the way, because of the problem, like just turn [automated] patching on.' Most IT providers come from a legacy of thinking everything has to be tested, constantly checked. Where do you fall in this idea of -- particularly for smaller companies like the typical SMB -- shouldn't they just turn autopatching on? Isn't that better?
Murchison: I'm going to have a weird answer. I was a network engineer before I got into hacking. I have a natural reaction that the IT guys have, which is, 'until my Windows 11 computer can update and not break shit.' … Think of manufacturers. There are so many industries where a patch takes them down. It can do as much harm as good. I'm a firm believer in continuous vulnerability scanning in a regular patching platform. If I saw a better track record of autopatching not breaking stuff, then maybe I'd support it. But I can tell you, we have issues once in a while in our cloud where we run a really modern stack, and Amazon might auto patch something for us that causes us problems.
I think there's just too many moving parts to send it. Now, if you're just talking about like auto Windows update on like an attorney's office that doesn't have manufacturing or any sort of weird technologies, yeah, I think it'd be fine. And I actually think Microsoft is working on some really cool new patching capabilities via their Graph API, if I've heard it right, that are outside of the Windows update. And I think that might open the door for a little more flexibility.
Sobel: I'll let you split the difference on that because, in fact, when I think of that, I do actually think of that typical small little lawyer. And I just sort of say, 'I think that I'll just turn the thing on automatically.'
Murchison: That one's probably good. When I look at our customer base, we do large enterprise, too, but it is like everything from a tiny five-person property management shop to a lot of manufacturers, logistics and people that really will go down hard if the patch messes up.
Sobel: And I think, by the way, this is where the expertise comes in. But it isn't a one size fits all. And I am advocating, at least, for many network engineers to go, 'Look, in some of these cases, autopatching is better.' There are use cases at the low end where just make that automatic and don't think about it because that's the bigger risk for them.
Murchison: I think so. But, you know, also when I look at what burns most companies. … It's funny, if you read the Verizon breach report, it's phishing, phishing, phishing, phishing. We do not see phishing as the number one way you get ransomware. It's the number one way you're going to detect malware because it's coming in through there all the time. And I think that skews the results. What we still see nonstop, [Remote Desktop Protocol] opening internet, a lack of patching on your network and security appliances. That's a huge area. Running a [demilitarized zone], they should be dead. Everyone screws them up. They screw up the rules. It's hard to write the firewall rules right. Maybe autopatching is best on anything sitting live on the boundary internet-facing.
When it comes to the inside, the reality is it's rare that we see exploits thrown inside the network. Usually, it's a way to get in. I'll tell you the area I totally believe in autopatching is in the browser. I'm sure hackers would disagree, but my opinion is the holy grail exploit, the one if you could pick any exploit you want -- it's not a firewall exploit because everyone runs a different firewall -- it'd be like an Edge, Chrome or Firefox remote code execution technique. If I can give them an ad or something or whatever exploit and get access to their system, that is the area. That should be, as soon as a new update comes out, roll it because that has less effects than a system patch -- because system patches just aren't exploited that much inside the network. I would focus on boundary.
Sobel: That's some great advice. I'm going to give one more targeted technical and then move this back into regulation. What about the idea of just blocking countries? The vast majority of small businesses, which are most businesses out there, they're not doing business with a long list of countries that just shouldn't even have access. Why aren't the default rules set to just block a whole bunch of countries from even coming in? Would that make more sense?
Murchison: Yes. Beats the hell of me why it's not. It should be just by default, period. Now, if someone really wants, it knocks out a certain amount of low-hanging fruit. There's no cost to doing it, and there's really no drawback for not doing it. I 100% agree with you, Dave. There's no question, those should be blocked default. I will caveat that with, we stopped the nation-state attack on one of our non-MSP customers the week of Thanksgiving. This was definitely in relation to a political move we made as a country. That night, we had guys come in. This entity was running two next-gen [antivirus endpoint detection and response] in addition to us, and the bad guys evaded both of those, no problem. Not a single alert, but they used residential proxy infrastructure in the U.S. to come through the United States to hide their tracks. SolarWinds stuff did that, too. And any legit operator will do that. It's not a cure-all, but it's a definite, do it.
Sobel: And that's what I'm getting at, is just like, 'Hey, if we can reduce our attack surfaces, let's do the simple things,' knowing, again, for the typical small, midsize company, they're not going to win against a nation-state. Let's just do that. Let me pivot then a little bit to regulation and the landscape. What's your take on the need for more formal rules of engagement around cyber, like similar to the Geneva Convention or international laws on physical warfare? Because it feels like we don't have even law enforcement or diplomatic rules to latch onto here. What's your take on the need for these formal rules?
Murchison: That is a loaded question. First off, I don't believe any international norms, diplomatic engagement, when a country or nation-state is desperate enough, they don't care. I think we're seeing that in Ukraine right this second. Cluster munitions -- using them like crazy. White phosphorus -- using them. I don't think that actually really moves the needle, I think that feels good. I'll tell you the area where we need a lot more focus is. … Step one, everyone creates a compliance framework. And, when I read these, half the compliance frameworks out there, they're written clearly from someone that was never an offensive practitioner or even a hardcore defensive. And they've written all post-breach lessons learned. Like you must have these logs so we can figure out what data was stolen. But, at the end of the day, these things should be increasing your capacity to not get breached in the first place or, if you do, nullify it before the adversary completes their objectives.
This is why I'm a huge fan of the [Center for Internet Security's] benchmarks and controls because they're actually prescriptive. They tell you to configure the following settings to reduce your attack surface. Whether it's by regulation or the market drives it, I think this is something where MSPs should really be looking to adopt it because, honestly, it's just smart business. Anything you can do to reduce your chances as an MSP owner from your customer getting smashed and then blaming you, which hurts your insurance and your whole business model, I would do it.
I'll tell you the other area -- and others might not agree with me -- that I think needs some tightening up. We have an insurance agency, right? We had to get licensed and fingerprinted in all 50 states to resell someone else's insurance policy. Yet, I can go write an exploit proof-of-concept code, throw it up on GitHub and get all the congratulations from the security community, no license, no regulation. People will do bug bounties. One of the scariest things -- and I agree, we need bug bounty stuff -- but one of the things that always makes me nervous is when you have vigilante entities out there that are doing pen testing without your request, which we all know it's kind of like, 'Do it well.' You have to cross that line sometimes and actually execute code. When they do that and they're like, 'Oh my gosh. We have something bad,' they call the company. Then, they make scanners that try to find other vulnerable victims. Well, to think that nation-states are not looking and have honeypots out there looking for these people with new exploit techniques, you're crazy. They are. And that's a great way, in the interest of security, to actually let the cat out of the bag. And then, it takes it a step further. Now, you're arming maybe more competent and capable adversaries with your great new research findings. And then, you call the company, and then you [Common Vulnerabilities and Exposures] them when maybe there is no user involvement required. I think it's ethically corrupt, and I think it's not good for security because, if you can close a problem, there's no impact, no harm. It hasn't been taken advantage of. You're still operating as a defender with a bit of an information asymmetry advantage or that vendor. When you make it public, how many variant follow-on [attacks]? Also, like the Exchange thing had multiple. ... Once the community found out about it, you start seeing follow-on exploits. And so, that's why I think nuance matters, details matter. And we have to balance transparency with actually doing what's right for the end customer and vendor to keep the infrastructure secure so that it isn't all about just getting resume builders for the security research community. Because I think they play an incredibly legit role, but there's no security industry I've ever seen in the world with utter, complete 100% transparency.
Sobel: You've opened the door. My follow-on question is obvious. What's your take on what a vulnerability disclosure program should be like? What's the way that it should work for software vendors?
Murchison: In my opinion, if there is a vulnerability that requires end-user involvement, like the IT guy, the end customer, to apply a patch because it cannot be pushed out, you have to shout that from the rooftops. That has to be public, and it has to be everywhere. If you have a capability that was brought to you, there was no evidence that it was ever exploited, it was never known about a criminal group, I personally think the act of writing about that is doing nothing but educating your adversaries on a potential weakness. And, if it was covered up, I think everyone should keep their mouth shut. I also think a formal bug bounty program, where you can work with the researchers, is really good.
Sobel: That's why I asked the question. If vulnerability disclosure is the front end, let me ask then what's your take on breach disclosure, which is essentially the back end of that. What's your take on breach disclosure notifications?
Murchison: Oh, I think that's a legal issue. And I think you absolutely have to do it, and each state has different rules on it. First off, that is something like you. ... Listen. You kind of judge people on what they do when no one's looking, right? And I think this is a perfect case, if you sweep it under the rug, that's an unethical move. You need to inform your customers. I would say the definition of breach is also, maybe there's a legal definition for it, but you know. … We stop stuff every day. Bad guys in network, he's going nowhere, he's snuffed out done. Should a vendor disclose that to the world? I don't know. I don't think that does anything for anyone but create concern or panic. If part of their code base was stolen, if their customer data was stolen, unequivocally, you have to.
Sobel: OK. I'm going to ask it a different way then. If I gave you the magic wand and you could cast a federal piece of regulation around breach disclosure, what would your version look like?
Murchison: I think my version would start with a definition of what we considered a breach. So, kind of along the lines of what I was just saying: loss of customer data, loss of [personally identifiable information], loss of source code or something that could enable follow-on attacks or something that actually disrupted operations. There was a malicious thing that disrupted operations and maybe left you not serviced at that point in time. That's something where there should be a requirement to disclose to your customer base and a requirement to disclose to the appropriate government agency.
About the author
Dave Sobel is host of the podcast The Business of Tech, co-host of the podcast Killing IT and authored the book Virtualization: Defined. Sobel is regarded as a leading expert in the delivery of technology services, with broad experience in both technology and business. He owned and operated an IT solution provider and MSP for more than a decade and has worked for vendors such as Level Platforms, GFI, LOGICnow and SolarWinds, leading community, event, marketing and product strategies, as well as M&A activities. Sobel has received multiple industry recognitions, including CRN Channel Chief, CRN UK A-List, Channel Futures Circle of Excellence winner, Channel Pro's 20/20 Visionaries and MSPmentor 250.