Background

Fuzz testing can be used to detect software programming flaws present in an application by submitting malformed input to the application as it executes.  Some programming flaws impact upon the security of an application by undermining the performance of controls, rendering the application vulnerable to attack.  Hence, the discovery of programming flaws can lead to the discovery of security vulnerabilities.  

Fuzz testing (like almost all run-time testing) does not require access to the source code, which makes it attractive to those who wish to assess the security of an application, but are unable to obtain access to the source code, such as end-users, corporate clients, security researchers and cyber criminals. 

Motivation

The author wanted to explore the value of fuzz testing from the point of view of a corporate client that intends to release software including a component developed by a third party, where the component source code is not available for review.     

Three case studies where conducted: two practical fuzz testing methodologies ('blind' data mutation and protocol analysis-based fuzzing) were employed to discover vulnerabilities in a commercial operating system, and a purposefully vulnerable web server, respectively.  A third case study involved the exploitation of a vulnerability discovered using fuzz testing, including the production of 'Proof of Concept' code.

Conclusions

It was found that fuzzing is a valid method for identifying programming flaws in software applications, but additional analysis is required to determine whether discovered flaws represented a security vulnerability.  

In order to better understand the analysis and ranking of errors discovered using fuzz testing, exploit code was developed based on a flaw discovered using fuzz testing.  It was found that the level of skill required to create such an exploit depends (largely) upon the nature of the specific programming flaw.  In the worst case (where user-controlled input values are passed to the instruction pointer register), the level of skill required to develop an exploit that permitted arbitrary code execution was minimal.

Due to the scale and range of input data accepted by all but the most simple of applications, fuzzing is not a practical method for detecting all flaws present in an application.  However, fuzzing should not be discounted since no current software security testing methodology is capable of discovering all present flaws, and fuzzing can offer benefits such as automation, scalability, and a low ratio of false-positives.