fuzz testing (fuzzing)

What is fuzz testing?

Fuzz testing (fuzzing) is a quality assurance technique used to discover coding errors or bugs and security loopholes in software, operating systems and networks. Fuzzing involves inputting massive amounts of random data, called fuzz, to the test subject in an attempt to make it crash.

If a security vulnerability is found, a software tool called a fuzzer is used to identify potential causes. Fuzz testing was originally developed by Barton Miller at the University of Wisconsin in 1989.

How does fuzz testing work?

Fuzzing is an application security testing technique that feeds invalid inputs to a software program to expose vulnerabilities. It is often a black box testing technique that is carried out without the knowledge of the subject program's internal structure.

Intentionally malformed, unexpected or completely random inputs are fed to the program to see how it responds. This approach reveals cybersecurity threats and vulnerabilities like memory leaks. Once identified, vulnerabilities can be patched so that threat actors can't use them to install malware, steal sensitive information or otherwise cause a data breach.

Fuzzing is typically carried out using automated software that generates new test cases and feeds the program inputs. It also detects if the program crashes, stalls or reveals other vulnerabilities. Some fuzz testing programs will mutate the input until it behaves unexpectedly, yielding a positive return from the test.

Programs that take structured input are usually the subject of fuzz testing. More nuanced fuzzers will feed the program semivalid input, which is valid enough for the program to accept during parsing but invalid enough that vulnerabilities are exposed.

Both paid and free open source software fuzzing tools are available.

Types of fuzz testing

Several different fuzz testing types are in use, including the following:

• Dumb fuzzers aren't aware of the input structure.
• Smart fuzzers are aware of the input structure.
• Mutation-based fuzz tests modify existing inputs into semivalid ones.
• Generation-based fuzz tests use inputs generated from scratch. This type of fuzzer doesn't need an existing body of inputs; it generates inputs based on the known input structure.
• Black box tests have no awareness of a program's structure and generate input at random. They check a program's basic functionality and are good for discovering surface-level bugs.
• White box tests are aware of a program's structure and design test inputs based on that knowledge. White box testing generally takes longer to generate inputs but is good for discovering bugs deep in the program.
• Gray box fuzzing has a partial understanding of computer program structure and is a mix of black and white box testing.
• Coverage-guided fuzz testing is a type of mutation-based testing that tracks the code coverage of fuzz test inputs and then assesses the program's behavior. The information from the initial test is used to which inputs to mutate for the most overall coverage.

Fuzz testing applications

Fuzzers are best used for discovering vulnerabilities that threat actors could exploit for various types of attacks, such as buffer overflows, denial of service, cross-site scripting and SQL injection. These are schemes malicious hackers use to wreak the greatest amount of havoc in the least possible time. Attackers use malicious code as malformed data inputs to manipulate a system, much the same way fuzz testing does.

Fuzz tests are used to test different data types in programming languages, including metadata, characters and binary sequences. Some methods of fuzz testing include the following:

• Protocol fuzzing tests protocol-level functionality using mutated network packets as input.
• File format fuzzing starts with a legitimate file sample and repeatedly mutates it until the program crashes trying to open the file.
• Application fuzzing tests the user interface, text inputs and command-line options.

Advantages of fuzz testing

Fuzz testing has a high benefit-to-cost ratio and often reveals defects that are overlooked when software is written and debugged. The following are some of the specific advantages to fuzz testing:

• Automation. Once a fuzzer application has been configured, in can use algorithms to run tests automatically, with little to no human intervention, saving money and time.
• Systematic approach. Fuzz is completely random input, so it is not subject to the bias of the software tester and can catch vulnerabilities that a human tester would miss.
• Good for closed systems. It's impossible to know the inner workings of a program or device built on a closed system, so random data testing is the only option. For example, in artificial intelligence and deep learning applications the exact relationship between inputs and outputs is impossible to discern.

Limitations of fuzz testing

Fuzz testing has some significant limitations, including the following:

• Missing attack types. Fuzz testing is less adept at discovering software security threats that don't cause program crashes based on malformed input. These types of threats include spyware, some viruses, worms, Trojans and keyloggers.
• Difficult setup. It takes significant coding expertise to manage and monitor fuzz tests, and to properly handle the bugs they identify.
• Limited scope. Most types of fuzz testing don't provide a complete picture of the security, quality and effectiveness of a program. They simply find bugs. Other types of software testing, such as functional testing and beta testing are needed, as well, in the software development process.

Several criteria go into software testing tool selection. Learn the seven questions to ask when selecting software testing tools.