How to avoid dangling pointers: Tiny programming errors leave serious security vulnerabilities

For years, many have said that there is no practical way to exploit a dangling pointer, a common application programming error. But these software bugs should no longer be thought of as simple quality-assurance problems. Michael Cobb explains how the threat has grown and reveals the techniques that can keep pointers from dangling helplessly in the wind.

In the past, fairly common programming errors called dangling pointers were often seen as quality control problems rather than security issues. In 2005, for example, a dangling pointer in Internet Information Server 5.1 was reported to Microsoft, but remained unpatched for two years, even though these kinds of bugs have the ability to crash a system and present a serious security threat.

Dangling pointers occur when a programmer creates, uses, and then frees an object in memory, but does not change the object's pointer value accordingly -- in this case, to a null pointer. Instead, the pointer inaccurately refers to the memory location of the de-allocated memory. The pointer is considered to be "dangling" since it points to memory that may no longer hold a valid object. Because the memory may contain completely different data, unpredictable behavior can result when a programmer mistakenly dereferences the pointer to access the object.

Experts at online risk management company Watchfire, recently acquired by IBM, have found a way to take control of dangling pointers and point them to a specific memory location. By sending a specially crafted URL to a server, researchers Jonathan Afek and Adi Sharabani discovered that they could crash a target machine and run their own shell code on it. Using this method, an attacker could remotely control or infect any machine that has a dangling pointer in one of its applications, in much the same way that an attacker could exploit a buffer overflow vulnerability.

And we have seen how damaging and effective buffer overflow attacks can be. Corrupted data structures, information leaks, privilege escalation and execution of malicious code are all possible exploits when an attacker meddles with computer memory. Although dangling pointers can probably be found in most mature applications, hackers are going to focus on Web server applications because they often contain a high volume of sensitive data.

Dangling pointers can be very difficult for programmers to locate since they often corrupt unrelated data or cause system instabilities long after they are created. The bugs are most prevalent in low-level languages such as C and C++. In higher-level languages like Java and C#, dangling pointers cannot occur. Here, pointers are handled automatically, and implicit garbage collection occurs when an object is eliminated. Developers using languages without some form of automatic handling of pointers need to reassess their application development processes. All pointers need to be set to a null pointer after use.

Smart pointers are popular data types that can better manage memory. A smart pointer typically uses reference counting to reclaim objects. Reference-counting mechanisms calculate the smart pointers that refer to the same object, and once the count equals zero, the object is deleted.

Some other defensive measures include the "tombstones method" and the "locks-and-keys method." Once an object is dead, an associated tombstone can provide automatic nullification of any pointers that may be referencing it. In a locks-and-keys scenario, pointer access is determined by comparing the two values of a pair of ordered keys. Other available products like Valgrind, a suite of debugging equipment, and the Boehm-Demers-Weiser garbage collector, a tool that cleans up code, can also help prevent dangling pointers.

Since dangling pointers can no longer be considered a benign problem, developers need to adjust software development processes so that dangling pointers are prevented from reaching production code. Because the problem may become too serious to leave until the quality control stage, protective measures should be placed earlier in the development lifecycle. With high-level languages, the process will be relatively straightforward. For those developing and maintaining programs in other languages, relying on the above-mentioned best practices is recommended.

More on this topic

  • Get an in-depth look at how Watchfire researchers exploited the dangling pointer.
  • For more information on emerging security threats, check out our special news coverage of Black Hat USA 2007.
  • Stumped on Web security? Send Michael Cobb your questions.

About the author:
Michael Cobb, CISSP-ISSAP is the founder and managing director of Cobweb Applications Ltd., a consultancy that offers IT training and support in data security and analysis. He co-authored the book IIS Security and has written numerous technical articles for leading IT publications. Mike is the guest instructor for several SearchSecurity.com Security Schools and, as a SearchSecurity.com site expert, answers user questions on application security and platform security.

Dig Deeper on Threats and vulnerabilities

Enterprise Desktop
Cloud Computing