How to mitigate Log4Shell, the Log4j vulnerability

What is Log4j and the Log4Shell vulnerability?

Logging plays an essential role in every application and IT system. A number of companies and applications -- from Twitter and Amazon to Microsoft and Minecraft -- use the Java-based Apache Log4j library to write events to log files.

Unfortunately, the Log4j library doesn't properly validate or escape input before logging it, an implementation defect called log injection. This defect means an unauthenticated remote attacker can send a specially crafted request to a server running a vulnerable version of Log4j -- versions 2.14.1 and below -- and launch a remote code execution attack to take control of the system.

Applications and systems log a lot of information, which means there are numerous vectors attackers can use to make a Log4j record the attack string. HTTP headers, such as User-Agent and X-Forwarded-For, the Uniform Resource Identifier and the request body are just a few examples.

How serious is Log4Shell?

The Log4Shell vulnerability, categorized as CVE-2021-44228, was first reported on Dec. 9, 2021. Attackers quickly took advantage of it because it is relatively easy to exploit. It was reportedly exploited prior to being disclosed to the public.

Just how serious is it? Quite serious. Cloudflare blocked 1.3 million attempts to use Log4Shell in just one hour on Dec. 14, 2021, while Check Point researchers have already identified more than 60 variations of the original exploit code.

Exploitation attempts detected so far in the wild seem to be coming from ransomware groups, access brokers, botnet herders and nation-backed advanced persistent threats, with malicious payloads of coin miners, remote access Trojans and ransomware.

The problem is so serious that the Cybersecurity and Infrastructure Security Agency (CISA) posted an emergency directive requiring federal civilian executive branch agencies to mitigate the problem by Dec. 24, 2021.

How to mitigate Log4Shell

Any company using Log4j version 2.14.1 or below -- or patched versions 2.15.0 and 2.16.0, which contain flaws -- is vulnerable to Log4Shell. The zero-day vulnerability has been patched by the Apache Logging Services Project. Enterprises have to deploy the security update -- Log4j version 2.17.0 -- but note that updating mission-critical applications takes time to ensure no functionality is broken or lost.

Another challenge with this flaw is it's not necessarily obvious where Log4j is being used. It could well be included in a third-party library or dependency, so systems can't be automatically assumed safe even if Java is not installed or not running in the process list. For example, the following could be true about Log4j:

• included in a Java Runtime Environment;
• runs only when triggered by another process, like cron; or
• used directly by a cloud service connected to the network.

If your company uses a software bill of materials (SBOM) for cybersecurity, check it for Log4j, and ensure anything using it upstream or downstream has patched the vulnerability. If your company needs help creating a cybersecurity SBOM, learn more about it here.

Be aware that the initial Log4shell fix was incomplete in certain nondefault configurations (CVE-2021-45046) and a denial-of-service vulnerability (CVE-2021-45105) has been fixed in version 2.17.0 for Java 8 users.

How to patch for Log4Shell

The only way to eliminate the vulnerability is to upgrade to a patched version of Log4j. Security teams need to start scrutinizing all systems and software for use of Log4j as a priority and apply the latest security patch for internet-facing software and devices as soon as possible. Any alerts generated by these components should be treated as a priority.

Vendor software

Security teams may have to wait for vendor guidance and updates for internally deployed software that is vulnerable, such as MuleSoft, Atlassian Bitbucket Server, Salesforce and AWS Lambda.

CISA is maintaining a list of affected vendors and products, and Apache's Log4j Security Vulnerabilities page details various workarounds to reduce exposure until patches can be applied.

In-house software

Applications built in-house should be scanned with a software composition analysis (SCA) tool or the CERT Coordination Center's CVE-2021-44228 scanner to detect applications that are vulnerable to Log4Shell.

Security researcher Florian Roth posted exploitation detection commands and rules to search for exploitation attempts against Log4j. Security researcher Rob Fuller created a list of hashes for vulnerable Log4j versions to help organizations search for them.

Technologies to mitigate the Log4j flaw

The most effective way to block malicious requests targeting Log4j is with a web application firewall (WAF). WAFs can compare request data against rules indicating CVE-2021-44228. Attackers will develop techniques and patterns to bypass these checks, however, so keep abreast of new developments and keep WAF rules up to date.

Note that obfuscation has already been detected, so signature-based detection alone won't be sufficient. Multilayered security is the only way to establish comprehensive protection against the numerous ways the Log4j vulnerability can be exploited. Monitor and inspect outbound traffic for signs of hosts responding to a Log4Shell packet or command-and-control callbacks.

Additional guidance

This article covers the key procedures to follow, but security teams should also research best practices for the specific systems and applications deployed in their environment.

Due to the serious nature of this vulnerability, various organizations have published guidance on how to deal with the threat of Log4Shell, including information from CISA and the Swiss Government Computer Emergency Response Team.

Consider reporting any Log4Shell compromises immediately to CISA and the FBI. Stay up to date on the latest Log4j news as further fixes and updates may be published as the Log4j library is further scrutinized by both malicious and ethical hackers.

Mitigating future flaws like Log4Shell

Modern applications and IT infrastructures are built using an enormous hierarchy of components and dependencies. By the time Log4Shell has been fully mitigated, the next severe vulnerability involving a widely used code library will be upon us.

To limit the risk from future vulnerabilities, security teams need to work with DevOps teams to ensure every component used within the IT environment is documented, along with an emergency plan of how they can be upgraded or patched. In-house development pipelines, meanwhile, need to integrate SCA and static and dynamic application security testing tools to prevent vulnerable libraries or code from being pushed to production releases.