3 ransomware detection techniques to catch an attack
It's not enough to protect a company's system from ransomware. Reduce damage from attacks with these three ransomware detection methods.
Try as they might, companies can't avoid ransomware forever. Eventually, attackers will get into an enterprise system. The goal then becomes detecting ransomware before it encrypts and exfiltrates business-critical data.
"The world has clearly recognized we cannot prevent every attack from happening," said Dave Gruber, analyst at Enterprise Strategy Group, a division of TechTarget. "The adversary is going to compromise our systems; they're going to get in. The race is to detect and stop attackers before anything happens."
When ransomware gets onto a company's system, it can cause serious damage, affecting the bottom line and public perception. By the time security teams see ransom demands, damage is done. Prevention is a critical piece of the battle against ransomware. But Allie Mellen, analyst at Forrester, pointed out that the detection and response activities in an IT security organization add a layer of protection. To protect against ransomware before it can make lateral moves in a system, companies need effective detection methods in place.
Security teams have plenty of options when it comes to malware detection techniques. Each technique falls into one of three types:
- signature-based methods
- behavior-based methods
- deception
Inside the 3 ransomware detection techniques
Ransomware detection involves using a mix of automation and malware analysis to discover malicious files early in the kill chain. But malware isn't always easy to find. Adversaries often hide ransomware within legitimate software to escape initial detection. Some software used includes PowerShell scripts, VBScript, Mimikatz and PsExec.
"The ultimate goal is to detect malicious activity, not necessarily to detect malware. The detection and analysis process is often assembling a series of what might be suspicious activities to determine whether anything malicious is actually happening," Gruber said.
1. Signature-based ransomware detection
Signature-based ransomware detection compares a ransomware sample hash to known signatures. It provides quick static analysis of files in an environment. Security platforms and antivirus software can capture data from within an executable to determine the likelihood that it is ransomware versus an authorized executable. Most antivirus software takes this step in a scan for malicious software.
Security teams can also use the Windows PowerShell cmdlet Get-FileHash or open source intelligence tools, such as VirusTotal, to get a file's hash. With current hashing algorithms, security professionals can compare a file's hash to known malware samples.
Signature-based ransomware detection techniques are a first level of defense. While useful at finding known threats, signature-based methods struggle to identify newer malware.
Attackers update their malware files to slip past detection. Adding a single byte to a file creates a new hash, decreasing the malicious software's detectability. In the first half of 2021, network security company SonicWall discovered 185,945 new malware variants, according to its "2021 Mid-Year Cyber Threat Report."
Still, signature-based detection is useful to identify older ransomware samples and "known good" files, said Mario de Boer, analyst at Gartner. It provides protection from ransomware campaigns that are general, rather than targeted, he said.

2. Behavior-based detection methods
Using behavior-based detection methods that examine new behaviors against historical data, security professionals and tools look for indicators of compromise by comparing recent behavior against average behavioral baselines. For example, is someone accessing a company desktop remotely from another state when the employee logged in from the office that same day?
Here are three such methods.
File system changes
Security teams should look for abnormal file executions, such as an overabundance of file renames. A few happen in a normal workday, but hundreds within a short amount of time should raise red flags.
Ransomware can stay hidden in systems for a while before executing. Therefore, security teams should also look for the creation of a file with larger entropy than an original file, as well as the enumeration and encryption of files.
Traffic analysis
Security teams should examine traffic for anomalies, such as whether any software is connecting to shady file-sharing sites and the time of such actions. Teams should also check whether the volume of traffic has recently increased and where it's going. Ransomware requires network connectivity to off-site servers to receive command and control instructions and to exchange decryption keys.
While useful, this detection method does yield false positives and requires analysis time. Also, attackers might use legitimate file-sharing sites, allowlisted by the infected company, to fly under the radar.
API calls
A third behavior-based method security teams can use is examining API calls. What commands are files executing? Are any suspicious? For example, spyware and keyloggers use GetWindowDC to capture information from an entire window. Or they use the IsDebuggerPresent to see if a debugger is active on a system.
Another ransomware ploy is to use GetTickCount to see how long a system has been on, to the millisecond. A short period of time may indicate that the ransomware is within a VM, and so it doesn't execute any malicious actions to prevent detection.
3. Deception-based detection
Tricking adversaries is the third ransomware detection technique. The most common example is to create a honeypot. This file repository or server is a decoy or bait for attackers. Normal users do not touch this server, so if it sees activity, the odds are good it's an attack.
Taking a layered anti-ransomware approach
Using multiple ransomware detection techniques together offers security teams a better chance to detect and monitor a ransomware attack -- and isolate it before it gets too far into a system.
"As modern attacks are becoming complex and easily bypass basic techniques, it is evident no single technique can address all use cases," de Boer said.
As such, companies need to do more than just install and run antivirus software. Alongside a combination of ransomware detection techniques, security teams should also look for attacks entering through the front door. Insider threats, such as credential reuse and social engineering, often give adversaries access to a system.
Companies need to take ransomware seriously. Ransomware payments are up 82% from 2020, according to data from Palo Alto Networks. Use best practices to train employees about the different ransomware risks. Teach infosec pros the Mitre ATT&CK framework, which provides tactics, techniques and procedures that adversaries use. With this knowledge, security teams can determine the company's strengths and weaknesses and improve systems accordingly.