Types of testing

Testing can be a manual or automated process, and there are advantages and disadvantages to each approach. When I test an application, I try to employ a healthy combination of both. Although this can be slightly more time consuming, being thorough tends to be more important than being expeditious.

With automated testing, you’re able to iterate through more tests than is reasonable with manual testing. You’re also able to execute your test suite as often as necessary and with more consistency. Tedium can adversely affect the testing process when manual steps must be repeated incessantly, and this can add a degree of unwanted inconsistency.

Automated testing has its own challenges, and there will always be special cases that require manual intervention. Interpreting instructions, solving CAPTCHAs, and adapting to change are difficult tasks to accomplish with automated testing alone, but manual testing makes these tasks simple. As with most things, try to use the best tool for the job, and the proper balance between manual and automated testing depends upon many factors, including the availability of necessary resources.

Puzzle Pieces

Although this article focuses on testing, it should not be considered a perfect solution. A complete security strategy also involves auditing code, processes, and people.

Auditing code, sometimes described as white box testing, is one of the most time-consuming but thorough pieces of the security puzzle. By closely inspecting your code for security risks and vulnerabilities, it’s possible to identify problems that are difficult to discover or exploit without access to the source code. It’s also possible to identify weaknesses in the design that can lead to problems in the future.

There are many processes involved in a live web app that can easily go unnoticed. What happens when someone discovers a security vulnerability? Is it easy to find the necessary contact information on the website? Is there a protocol in place for addressing the issue(s) and responding to the reporter? What happens if the vulnerability is exploited? These questions and more are much easier to answer before they are asked, and a lack of planning can lead to problems in the future, regardless of how secure your application(s) may be.

Auditing people involves testing those who are involved in the care and upkeep of the web app. A perfect example is a customer support representative with special access to application data. Have all representatives been properly trained to detect and avoid social engineering attempts? How certain are you that an impostor cannot gain access to a customer’s sensitive data with a simple phone call? Customer service representatives are just one example, but people are often the weakest link, so it makes sense to ensure they are properly trained and tested, too.

The type of testing this article addresses is often described as black box testing, and it’s a complimentary technique to code auditing. Black box testing involves using the app in its native environment, whether production or a test environment designed to mirror production. This process can help to verify or correct any assumptions about the environment, because static code can only paint part of the overall picture.

Testing Challenges

There are a few challenges involved in black box testing. Sometimes overcoming these challenges simply requires some manual intervention, but other times inspection of the source code is necessary. One of the best examples of the latter is a trivial backdoor:

1. <?php
2.  
3. if ($_GET['admin'] == 'yes') {
4.   /* User is an administrator. */
5. }
6.  
7. ?>

This particular example indicates a serious flaw, where anyone who accesses a URL with ?admin=yes is authenticated as an administrator. Although it is quite easy to spot, hopefully you can see how difficult it can be to identify such a flaw without access to the source code.

Other challenges that you’re sure to encounter can be easily solved with some manual intervention. For example, sometimes access to a particular page requires you to play by the rules on all of the pages prior, and these rules might be provided as on-screen instructions. If there is a vulnerability on the last page in a series, an automated test can potentially miss it by failing to reach that particular page. Another example is a CAPTCHA. Because CAPTCHAs are specifically designed to require manual intervention, it is difficult to automate a test that can bypass them.

Testing Sources

As you test, it’s important to categorize your tests, so that you can be sure to be as thorough and efficient as possible. One good category to document is the various sources of input. Although there are many aspects to a secure web app, the proper handling of input is one of the most crucial. Your tests should at least involve $_GET, $_POST, $_COOKIE, and any HTTP headers such as Referer that are used.

Testing $_GET is arguably the easiest source to test, because you can simply manipulate the URL.

Testing $_POST is a bit more involved, but still pretty straightforward. A common technique is to copy and paste the form locally, changing the action to an absolute URL. This works just fine, but it is easily defeated with Referer checking (a weak and useless safeguard) and is slightly cumbersome. A better approach is to write some code to send the request:

1. <?php
2.  
3. require 'HTTP/Request.php';
4.  
5. $request = new HTTP_Request('http://host/login.php');
6. $request->setMethod(HTTP_REQUEST_METHOD_POST);
7. $request->addPostData('username', "chris' --");
8. $request->addPostData('password', '');
9. $request->sendRequest();
10.  
11. ?>

Once you’ve written some code, it’s easy to enumerate through many different tests.

Yet another option is to use a browser extension such as Tamper Data for Firefox to modify the HTTP requests sent by your browser. HTTP proxies such as Burp proxy are also easy to work with, and these are browser-independent. These two techniques can also be used to test HTTP headers such as Referer.

One of my favorite techniques for testing cookies is to use bookmarklets, tiny scripts that use the javascript scheme and can therefore be bookmarked like any other URL:

1. javascript:document.cookie = prompt(document.cookie,document.cookie)

This particular example is the most generic, as it lets you modify document.cookie in its entirety. It’s easy to create a collection of these, and a simple click can prepare you for a valuable test.

Testing Context

Knowing how to test various sources is only a small part of your task. For each source, you want to enumerate through many different attacks. These attacks should take two things into consideration:

1. Context
2. Attack vectors

Testing context involves making some guesses about how the data is being used. For example, to test for XSS, it’s not enough to simply craft a single XSS exploit. Depending upon how the source you’re testing is being used, it might be necessary to break out of a particular context:

1. <form action="<?php echo $_SERVER['PHP_SELF']; ?>">

Exploiting this vulnerability requires a URL such as the following:

1. http://host/script.php/">{xss}<"

The string {xss} is just a placeholder. The next section discusses XSS testing in more detail.

Another common example of testing context is breaking out of a quoted string in SQL:

1. SELECT count(*)
2. FROM users
3. WHERE username = '{$_POST['username']}'
4. AND password = '...'

If the source being tested is $_POST['username'], it’s necessary to inject a single quote in order to break out of the quoted string:

1. chris' /*

Testing Attack Vectors

There are many different attack vectors, some of which I’ve covered here in Security Corner. XSS continues to be one of the most common vulnerabilities, so it should definitely be on your list. There are two primary types of XSS for which your testing must account:

1. Reflected
2. Persisted

Reflected XSS is the easiest to discover, and it is best described by example. Consider a URL such as the following:

1. http://host/error.php?message=We+apologize+for+the+error.

A simple test is to visit the same URL with a different message:

1. http://host/error.php?message=%3Cxss%3E

If <xss> is in the response, it's very likely that this particular page is vulnerable to XSS. Reflected XSS can be tested by simply examining the responses to various injections, searching for output that is not properly escaped.

Consider an entirely different scenario where you are registering as a user. If the username you provide contains your XSS test, but the username is not in the response, how do you know if the test is successful? After all, perhaps there is a vulnerability, but it’s on an entirely different page where your username is displayed. Alternatively, perhaps the username is in the response and escaped properly. Can you be sure the test is successful if the username is persisted? There might still be another page that is vulnerable.

Persisted XSS is much more difficult to test, but a relatively good technique is to use the following as your test:

1. <script src="http://host/stats.php"></script>

Assuming host is your own host, stats.php is a script that logs $_SERVER['HTTP_REFERER'] and $_SERVER['HTTP_USER_AGENT']. Combined, these indicate where the vulnerability exists and which browser(s) can be exploited. With this particular attack persisted, all you must do is visit every page in your application with each browser you wish to test. This is easy with a tool such as Selenium.

Until Next Time…

Security testing is a pretty big topic, and this article has barely scratched the surface, but hopefully you’re well on your way to coming up with your own techniques and strategies. If you have some helpful tips or discoveries related to this topic, I’d love to hear about them.

Until next month, be safe.