Unintentional backdoors are either introduced due to programming errors or an unawareness of the threat that backdoors present. For example, programmers routinely and purposefully insert a backdoor into an application, in order to access that application later for remote troubleshooting.
As the complexity of modern software applications increases, along with the use of components assembled from reusable binary components, backdoors can easily circumvent even the best of QA cycles. The need for an accurate and complete approach to software security testing is more pressing than ever.
At the same time, applications have become a target-rich environment for those seeking monetary gain through the misuse of personal information. These trends have increased the occurrence of backdoors being inserted into applications.
Binary-level application testing or compiled-code analysis makes it possible to examine the entire application in its final delivered form. It enables the most complete detection of backdoors that most often can only be found in application binaries.
An application backdoor is a segment of code that provides an alternative, non-standard entry into an application and can permit functionality not intended by the application's designers. Application backdoors can be inserted unintentionally, intentionally for good reasons, or intentionally for malicious reasons. A common characteristic of backdoors is that they are often planted into an application with the intent that they will be exploited at a later point in time.
While backdoors are becoming more common, their origins are varied. Backdoors are not always introduced with malicious intent. For example, conscientious programmers may build a backdoor into an application for the purpose of remote troubleshooting, an activity that is more common in today's dispersed, interconnected application environments.
Less-careful programmers can unintentionally introduce backdoors by re-using code that has not been checked, or simply by being unaware of the backdoor exploitability factor in certain types of code.
Undetected backdoors, regardless of their origin or intent, can give attackers access to data, including customer and company data, and allow them to steal or manipulate that data. This obviously exposes the application developers and their companies to potentially serious risks.
In the past, manual code reviews or automated reviews of source code were used in an effort to detect backdoors. However, these efforts have resulted in limited success due to both the high cost of manual reviews and the high false negative rate of source code analysis tools. Source code analysis tools miss significant portions of the software such as third-party libraries, API calls and code injected via the tool chain during compile time, making the technique unreliable at best.
Recently, new technology has enabled organisations to overcome the limitations of source code testing, by using static binary analysis to provide a complete way to cover application security. Static binary analysis of applications represents a superset of source code analysis.
Three types of backdoors
System backdoors allow access to data and processes at the operating system root level. They are typically created by an attacker who has compromised a system in order to retain access to the system later on. Examples of system backdoors include rootkits, remote access software, malware such as Trojans or bots and deliberate system misconfiguration.
Crypto backdoors are intentionally designed weaknesses in a cryptosubsystem for particular keys or messages that will allow the attacker to gain access to clear-text messages. Application backdoors are sections of legitimate software modified to bypass security mechanisms under certain conditions.
Application backdoors are sections of legitimate software modified to bypass security mechanisms under certain conditions. These legitimate programs are meant to be installed and running on a system with the full knowledge and approval of the system operator.
Application backdoors can result in the compromise of the data and transactions performed by an application and may also result in a compromise of the complete system.
Special credentials
In these cases, the attacker inserts logic and special access credentials into the program code, typically in the form of a special user name, password, and password hash or key. One company had such vulnerability in its popular commercial database program; it went undetected for seven years.
This vulnerability could have been successfully detected using a combination of static detection strategies, including analysis of variables that look like usernames or passwords, identifying static variables that look like hashes, and identifying static variables that look like cryptographic keys.
Hidden functionality
Attackers use these methods to issue commands or authenticate without needing to perform the designed authentication procedure. More advanced attackers even include a special check so that there is some protection from anyone using the backdoor. In 2007, this type of backdoor was inserted in a popular blog publishing program, WordPress, which led to a recall of an entire version of the software.
Techniques for identifying this vulnerability should include inspecting the application for command injection vulnerabilities, recognising patterns in scripting languages, comparing server-side code with client-side code, identifying potential operating system command injection vectors, and identifying static variables that look like application commands.
Unintended network activity
The attacker can exploit networks by listening on undocumented ports, making outbound connections to establish a command/ control channel, or leaking sensitive information over the network. In many cases, these backdoors are strengthened with rootkits, which are of further concern.
In 2002, a backdoor of this type was inserted in the source code distribution of a popular network sniffer. It established a connection to an IP address and listened for commands.
It is possible to detect this type of activity using dynamic network utility testing, but only if the specific run conditions, under which the program will activate, are duplicated.
A number of static analysis techniques can be used to detect this code, including identifying inbound and outbound connections, analysing network activity that references a hard-coded IP address or port, identifying potential information leaks, analysing data flows, examining import tables, and more. The goal is to identify anomalies, then analyse the binary code in more depth.
Parameters that assign certain privileges to a process, influence task scheduling, or restrict operations on memory pages represent a higher security risk. In 2003, an attacker attempted to insert a backdoor into the Linux kernel. Fortunately, the backdoor was noticed and removed before users were affected.
The method the attacker used wasn't successful in this case, but it revealed a very subtle technique involving just a couple of lines of code that were activated when a particular process was initiated.
The technique used by this attacker could be mistaken for a common, usually benign, programming flaw - using assign instead of compare. Static analysis could be used to identify all instances of this particular behaviour, taking into account the instances where this string of code is intentional.
Automated source code analysis
Several tools are now commercially available to review large and complex applications without manually going through the code line-by-line. These tools are good at finding secure coding flaws, such as memory usage errors, input validation errors, or mishandled error conditions that an attacker can use to compromise the software.
However, they require access to source code, which many not always be feasible and doesn't take into consideration backdoors that may be injected during compile time, via APIs or hidden in third party libraries. Additionally, this method does not detect most types of application backdoors, precisely because they are designed to be hidden in the source code and only become exploitable after compile time.
Static binary, byte-code or compiled analysis examines a compiled form of an application or component to create a complete picture of real-world vulnerabilities. It is important to understand how its two main aspects: binary and static separate it from alternative methods.
Commercial off-the-shelf (COTS) programs can be a problem. An application security expert recently analyzed 100 programs. He randomly selected 100 COTS and open source application packages, analyzing those using reverse-engineering tools. Upon tabulating the results, he found that:
- 79 packages had dead code;
- 23 packages had unwanted code;
- 89 packages had suspicious behaviours;
- 76 packages had possible maliciou code.
Of these:
- 21 packages had known worms, Trojans, rootkits, etc.
- 69 packages had possible worms, Trojans, rootkits, etc.
Whether intentional or not, security vulnerabilities present in an application can leave an enterprise or government organisation open to serious legal or commercial risk. Given the complexity, age, and varied pedigree of code that characterises the typical application, source code analysis is incomplete.
Once backdoors and malicious code are inserted, they are impossible to detect without binary application testing analysis. On-demand, static binary application testing for backdoors gives developers and enterprises a complete solution to testing their applications for critical security vulnerabilities.
