About the Author

Chris Shiflett

Hi, I’m Chris: web craftsman, community leader, husband, father, and partner at Fictive Kin.


All posts for Dec 2005

Happy Holidays

Essential PHP Security: Forms and URLs

The sample chapter of Essential PHP Security for MySQL's Developer Zone is now available:

This chapter discusses form processing and the most common types of attacks that you need to be aware of when dealing with data from forms and URLs. You will learn about attacks such as cross-site scripting (XSS) and cross-site request forgeries (CSRF), as well as how to spoof forms and raw HTTP requests manually. By the end of the chapter, you will not only see examples of these attacks, but also what practices you can employ to help prevent them.

I hope you enjoy it. :-)

Google XSS Example

In the comments to my previous blog post, Ivo Jansch asks:

To be able to comprehend how this may affect my website, could you explain how this could be exploited, even though you cannot demonstrate it?

Rather than offer another vague answer, I decided to provide a very simple proof of concept that demonstrates how character encoding inconsistencies can bite you. Google's vulnerability has of course been fixed, but with a simple PHP script, we can reproduce the situation:

<?php 
 
header('Content-Type: text/html; charset=UTF-7'); 
 
$string = "<script>alert('XSS');</script>"; 
$string = mb_convert_encoding($string, 'UTF-7'); 
 
echo htmlentities($string); 
 
?>

If you run this PHP script, you should see a popup window:

Although the output is escaped with htmlentities(), the JavaScript is still executed by the browser.

The example attack is a UTF-7 string (I just use mb_convert_encoding() for this demonstration), and the browser interprets the page as UTF-7 due to the Content-Type header. Internet Explorer makes this assumption automatically (thus, you can remove the explicit header() call), but this example should work in any browser.

Hopefully developers will begin to appreciate the necessity of character encoding consistency. If anyone ever tries to claim that it doesn't matter, you can point them here. :-)

Google's XSS Vulnerability

The recent cross-site scripting (XSS) vulnerability discovered in Google perfectly illustrates why character encoding matters. This example demonstrates how to use PHP's htmlentities() function with the optional third argument that indicates the character encoding:

<?php 
 
$html = array(); 
 
$html['username'] = htmlentities($clean['username'], 
                                 ENT_QUOTES,
                                 'UTF-8'); 
 
echo "<p>Welcome back, {$html['username']}.</p>"; 
 
?>

The example uses UTF-8, so this should be indicated in the Content-Type header:

Content-Type: text/html; charset=UTF-8

Researchers at Watchfire realized that Google does not indicate the character encoding. They also realized that you can visit a URL such as the following to get data that you send returned in the content of the response:

http://google.com/url?EVIL

You will see the following:

Forbidden

Your client does not have permission to get URL /url?EVIL from this server.

Google fails to handle malicious attacks that use UTF-7, so all an attacker must do is target a browser that will interpret Google's response as a UTF-7 resource. Because Google does not indicate the character encoding in its Content-Type entity header, this is possible.

Unfortunately for Internet Explorer users (and Google), there is an auto select option for encoding that, if set, will interpret a resource as UTF-7 if it finds a UTF-7 character in the first 4096 bytes. Because Google's response is so small, the danger is clear.

The moral of the story is that you should always ensure character encoding consistency between your escaping function and the remote system to which you're sending data. In other words, specify the character encoding in htmlentities(), use mysql_real_escape_string() (which handles this for you), etc.

Google corrected this flaw earlier this month.

Power PHP Testing

The tutorial that Geoff Young and I gave at ApacheCon has sparked some discussion (mostly via email) that I think will lead to better testing tools for PHP developers. A PDF of our slides is now available:

Geoff also has some tarballs available that let you test a very simple PHP library (functions.inc) with Apache-Test, Simple-Test, PHPUnit, and phpt:

One of the tarballs demonstrates how to use the Simple-Test testing library within the Apache-Test testing framework. This is thanks to the work of Mike Lively, who has documented his work in his blog:

His most recent tarball contains everything you need to use these two tools together.

After we mentioned TAP (Test Anything Protocol) to Sebastian Bergmann, he added TAP support to PHPUnit. Now, at least conceptually, you can also use PHPUnit to write your tests. This gives PHP developers three choices for writing tests within the Apache-Test framework:

  • The bundled PHP port of Test::More
  • Simple-Test with the TAP Reporter
  • PHPUnit with the TAP Logger

We've also been discussing the various advantages and disadvantages of each tool as well as how we might be able to help make testing easier for PHP developers. One of the perspectives I've been highlighting is best stated by Matthew Weier O'Phinney in his blog:

I find writing the tests tedious. In Simple-Test, as in PHPUnit, you need to create a class that sets up the testing harness, and then you create a method for each test you wish to run, and so on. I found it incredibly time consuming.

I don't think Matthew knew about Apache-Test's Test::More library (it's as simple and straightforward as phpt tests), but this illustrates one of the disadvantages of testing tools that require a lot of overhead - they raise the barrier of entry (and they don't really fit in with the "PHP way" of solving problems as simply and directly as possible). This was one of the reasons why I ported Perl's Test::More library to PHP - it's very simple and doesn't get in your way. It also works with many mature testing tools already available (such as Apache-Test), because it's TAP-compliant.

Sebastian says he's working on adding phpt support to PHPUnit in an attempt to lower the barrier of entry.

On a related note, it looks like there will be a talk about this stuff at the 2006 PHP Quebec conference called Using Test::Harness to Test PHP Applications:

The Perl community has long had a very powerful unit-testing tool available: Test::Harness and friends. It uses what the Perl people call TAP - the Test Anything Protocol. I've used the Perl framework myself to verify correct behaviour in Perl modules and the Apache Web server. It came as a surprise to me that there was apparently no port of the technology to PHP, and so I've done some work toward correcting that. This session will include an introduction to the technology, a description of the implementation, and examples of how it can be used to test PHP applications.

I think Ken's interest highlights the usefulness of these tools. (He's been notified that this work has already been done, so hopefully it can save him some wasted time.) With any luck, Ken will not only talk about Test::More for PHP but also the TAP support that is now available in Simple-Test and PHPUnit.

JApacheCon Wrapup

Despite the heavy emphasis on Java at this year's ApacheCon, I still enjoyed the conference and learned a lot. Michael Radwin was giving a talk that looked very interesting, but my travel plans prevented me from being able to attend. Although it's not nearly as good as seeing him live, you can view his slides online:

I got to see Rasmus Lerdorf's talk, and you can also view his slides online:

Rasmus actually covered many topics, including Ajax, Flickr, and Yahoo Maps. It felt like he was just having fun and talking about the cool things he's been playing with recently. You can see more of his toys on his toys page, which is the closest thing he has to a blog.

Andrei Zmievski spoke about Unicode. He very clearly presents the problems faced in the work he's doing, which helps you understand and appreciate the solution.

Adam Trachtenberg spoke about web services, but I also missed his talk. He hasn't posted the slides yet, but hopefully he will soon - he's already been bragging about finishing them early. :-)

My favorite talk was Christian Wenz's talk on web application security. It's always interesting to see other people's approach to this topic, particularly the audience's reaction. The audience simply loved his talk. He covered several of the most common web application vulnerabilities, and the talk was mostly driven by live demonstrations. Rather than focusing on principles and theory, he demonstrated realistic attacks. It was a bit disorganized, but very entertaining.

You can view my ApacheCon 2005 gallery online.

CodeSnipers.com Interview

CodeSnipers.com recently interviewed me, and that interview is now available on their web site. Topics range from my book to my involvement in the PHP community.

I've been interviewed before, but never about myself, so this was a new experience for me. Thanks to Keith Casey for the opportunity.

JApacheCon

I'm at JApacheCon (ApacheCon that has been infested with Java) this week in sunny San Diego. On Sunday, Geoff Young and I gave our tutorial, Power PHP Testing, which went really well. Most of the attendees had PHP experience and no testing experience, so it was a perfect fit.

We covered testing theory as well as some practical examples using phpt, Simple-Test, PHPUnit, and (of course) Apache-Test. We have tarballs for each framework that provide everything you need (I'll link to these in another post), including a Makefile so that make test runs your test suite.

The challenge with being the authors of one of the testing frameworks is that we are obviously biased. (This is stated in the abstract.) Our approach was to try to make every testing framework shine as much as possible, because the purpose of the talk is to encourage testing in the PHP community, regardless of which tool is used. I'll probably elaborate later, but here are some of the things we did for each framework:

  • Simple-Test - Because the default reporter (reporter.php) is an HTML reporter, automated testing is difficult. (You'd have to write your own HTTP client to aggregate the results.) Luckily, Mike Lively has created a TAP reporter (tap-reporter.php), so we used this in our demonstration (and tarball).
  • PHPUnit - PHPUnit has a number of dependencies (which is less of a problem if you already have a fairly complete PEAR installation), so we bundled everything you need. We also wrote our own phpunit tool to run the tests (this is executed when you type make test), and it modifies include_path to resolve the inconsistent ways that dependencies are included.
  • phpt - Executing phpt tests is actually very straightforward, so all we did was add the test target to the Makefile to execute the proper command. We did add an assertNoUnwantedPuke target to our Makefile to clean up the mess that is created whenever tests fail.
  • Apache-Test - Because Apache-Test already supports automated testing with make test, no changes were necessary. However, we did create a tarball to demonstrate using Simple-Test (instead of test-more.php) within the Apache-Test framework.

The biggest difference between the various frameworks is how failures are handled, and this is where Apache-Test shines. With some enhancements to the TAP reporter for Simple-Test, however, I think it can be just as good. I'll probably blog about this separately after speaking with the authors of the other frameworks, because I want to make sure that we're highlighting the advantages and disadvantages of each framework fairly.

The conference has been great for the hallway conversations and the opportunity to meet and work with other speakers and attendees, but the interesting sessions are few and far between. Andrei Zmievski's recent comment, "2:00 is the next real session," sums this up nicely.

PHPSecurity.org Launches

PHPSecurity.org, the companion web site for my new book, Essential PHP Security, is now online. Many thanks to Amy Hoy for the excellent design!

I've included the table of contents, the (unfortunate) errata, some reviews, and the code repository.

Some of the examples in the code repository might raise ethical concerns, but I tried to be very careful not to provide full-featured tools that script kiddies can use. For example, the session injection script only lets you modify strings and is split into two separate examples (edit.php and inject.php), and the script to let you browse the filesystem is very basic. I've considered enhancing these to make them more useful (and more robust), but I fear they would be misused. What do you think?

The reviews have been very good. I'm happy to see that so many people appreciate the book's small size and focus. Thanks to everyone who has taken the time to record your thoughts. I really appreciate it!

Several people have asked how the book is selling, and I honestly don't know. It has frequently been on Technorati's Popular Books list as well as in the top 10 PHP books on Amazon. I haven't found a really good site for tracking the Amazon Sales Rank, but Rankforest isn't bad (and they use PHP). Anyone have any better suggestions?

The sample chapter for MySQL's Developer Zone still hasn't been posted, but hopefully that will happen soon. Until then, you can read Chapter 4, Sessions and Cookies (PDF) or get your own copy. :-)

Zend Framework Webcast

I just finished listening to the Zend Framework Webcast, hosted by php|architect. The recording will be available soon, and I'll update this post to provide a link as soon as it is.

The core focus of the framework is Extreme Simplicity. In order to achieve this, it must be very easy to use. This doesn't just mean that it must be simple to write code - it also needs to be useful in existing PHP environments without the need for external libraries or custom modifications. In many cases, this means that features must be implemented in pure PHP, so that they're available everywhere. For example, the search component is a pure PHP implementation of Lucene, and the front controller doesn't rely on a sophisticated collection of mod_rewrite rules.

Another goal of the framework is to allow developers to use only what they need. If you just want to use the framework's nice search component, you can. You can pick and choose from among the framework's features without having to choose its way of doing everything.

However, this is a real framework and not just a collection of components. The framework will provide components that can be used separately, but it also provides the glue to bring them all together. In addition, the framework will be distributed in its entirety, so there's no need to worry about installing invidual components or resolving dependencies.

The following figure provides an overview of the framework (click on the image for a larger view):

Three components were demonstrated with use cases - ZActiveRecord, ZMail, and ZSearch.

ZActiveRecord, which is an implementation of Martin Fowler's Active Record pattern, provides a nice data interface. Results are returned as a ZActiveRecordCollection object, which implements the Iterator and ArrayAccess interfaces, so result sets behave like ordinary PHP arrays:

<?php 

class Person extends ZActiveRecord {}    

$people Person::findAll(array('nameFirst' => 'Daniel'));

foreach (
$people as $person)
{
    echo 
$person->nameFirst "\n";
}

?>

For small queries, there are various find() methods provided, such as findFirst():

<?php 

class Person extends ZActiveRecord {}

$zeev Person::findFirst(array('nameFirst' => 'Zeev'),
                                
'nameLast'  => 'Suraski'));

?>

For more complicated queries, you can use findBySql(), which allows you to provide your own SQL query. This is better than having a complicated query builder.

ZMail was demonstrated with a simple example (I've modified the email addresses out of courtesy):

<?php 

$mail 
= new ZMail();

$mail->setFrom('daniel@example.org''Daniel Kushner');
$mail->addTo('paul@example.org''Paul M. Jones');
$mail->addTo('matthew@example.org''Matthew Weier O\'Phinney');

$mail->setSubject('ZMail Demo');
$mail->setBodyText('Hi All, here is some text.');
$mail->setBodyHtml('<blink>HTML</blink>');

$mail->send();

?>

ZSearch is arguably one of the coolest features of the Zend Framework. It provides both a simple API and a more sophisticated query builder. For simple queries, just use the find() method:

<?php 

$index 
ZSearch::open('/tmp/index');

$hits $index->find('zend');

foreach (
$hits as $hit)
{
    echo 
$hit->getDocument()->getFieldValue('title') . "\n";
}

?>

The find() method also supports Google-like queries:

<?php 

$index 
ZSearch::open('/tmp/index');

$hits $index->find('zend php -java');

foreach (
$hits as $hit)
{
    echo 
$hit->getDocument()->getFieldValue('title') . "\n";
}

?>

As you can see, most of the objects implement the Iterator and ArrayAccess interfaces, so working with them is very easy. The idea is that the framework takes care of all of the complexity, so that the code you write is extremely simple. In a way, the Zend Framework makes PHP development as easy as SimpleXML makes XML.

PHP Magazine December Issue

PHP Magazine just published their December issue. The cover article is an introduction to design patterns by Robert Peake. My column, Guru Speak, discusses the interesting things you can do with output buffering.

My favorite output buffering trick isn't really a trick at all - it's a relatively new (PHP 4.3+) function called output_add_rewrite_var(). This function makes the otherwise tedious chore of rewriting URLs very easy. For example, if you decide you want to propagate an auth token to strengthen your session mechanism, it's very easy:

<?php 

output_add_rewrite_var
('auth''412e11');

?>

Here's a larger example that demonstrates what this does:

<?php 

output_add_rewrite_var
('auth''412e11');

?>
<a href="link.php">Click Here</a>
<form action="form.php" method="POST">
<input type="submit" />
</form>

PHP propagates the auth token in both the link and the form:

<a href="link.php?auth=412e11">Click Here</a> 
<form action="form.php" method="POST">
<input type="hidden" name="auth" value="412e11" />
<input type="submit" />
</form>