About the Author

Chris Shiflett is an author and speaker who leads the web application security practice at OmniTI.

Referer Buys You Nothing

Fri, 04 Feb 2005 at 07:28:03 GMT
23 Comments
References

I am very surprised at how often I see Referer checking being mentioned as a safeguard against form spoofing. I can't properly express how completely useless this is. I've even had people try to argue with me, convinced that this is a sound technique.

Consider a hypothetical form located at http://example.org/form.html:

<form action="/process.php" method="POST"> 
<input type="text" name="foo" /> 
<input type="submit" /> 
</form>

To spoof this form, an attacker sends an HTTP POST request to http://example.org/process.php. Assuming the developer who wrote process.php is relying on Referer checking to prevent form spoofing, guess what the expected value is? Does this really seem like a big secret? An attacker will get this right every single time.

If you want to do something useful, at least use some bit of information that isn't obvious. One example is to generate a secret token and include it in the form:

<?php 
 
$token = md5(uniqid(rand(), true)); 
$_SESSION['token'] = $token; 
 
?> 
 
<form action="/process.php" method="POST"> 
<input type="hidden" name="token" value="<?php echo $token; ?>" /> 
<input type="text" name="foo" /> 
<input type="submit" /> 
</form>

You can check this value in process.php, and it's not very easy to guess. In fact, the only person with a reasonable chance of knowing this value is the person you send it to.

About This Post

Referer Buys You Nothing was posted on Fri, 04 Feb 2005 at 07:28:03 GMT.

23 Comments

1. Mathieu said:

Still, even this token could be read by a script and sent along with other 'spoofed' data. It'd make the malicious script a little bit more complex (and possibly slower), but not by much.
Fri, 04 Feb 2005 at 08:07:25 GMT Link

2. Chris Shiflett said:

Show me an example exploit. I think you're misunderstanding something.
Fri, 04 Feb 2005 at 08:10:46 GMT Link

3. HarryFuecks said:

"I am very surprised at how often I see Referer checking being mentioned as a safeguard against form spoofing."

Makes me wonder if this is a result of browsers being too good at hiding users from the nature of HTTP? Perhaps an effective remedy would be a hands on tutorial of writing an HTTP client with something like PEAR::HTTP_Request (or even from scratch with fsockopen).

"I can't properly express how completely useless this is."

Great sentence ;)
Fri, 04 Feb 2005 at 08:11:49 GMT Link

4. ryan king said:

I certainly agree that referer checking is by no means sufficient for security, But, still, my code's all going to have referer checking in it because I'll take any little bit of help I can get.
Fri, 04 Feb 2005 at 08:20:23 GMT Link

5. Mathieu said:

Chris, you can easily write a script that'll first retrieve the page through GET => filter out the token => use that in a POST along with data you want to muck around with.
Fri, 04 Feb 2005 at 08:36:01 GMT Link

6. Alan Knowles said:

I decided to actually implement the javascript form mods today to try and reduce blog spamming. While not a perfect answer to this, it adds a level of customization to deal with the form autoposting that is so specific to a particular site that it may not be viable for a blog spammer to implement.

In my case this means changing the form tag to:

action="/dev/null"

onsubmit="this.action=this.getAttribute('real:post'); this.chk.value='ok'; return true;"

Add a hidden element:

<input name="chk" value="" type="hidden">

On the server side, I just check for the chk value that I decided to use. (it also aleviates the necesity to have session vars for all vistors.)

There is a finite limit though of what you can do without resorting to sessions and clever javascript checksums. It's almost impossible to solve for a determined post spammer, but does deter casual ones.
Fri, 04 Feb 2005 at 08:44:52 GMT Link

7. Vincent said:

From what I've noticed with spammers and session data is that they've grown smart to it and pass the session id in the URL or with the cookie header, so the only way around that is to 'session.use_only_cookies = 1' but this could block real users (are there still people with cookies disabled?) or to use the IP address in the session too (also causing problems for people on ISPs like AOL) as they seem to request the page with one proxy and then spam from multiple others

I don't know if its just the one spammer hitting my sites or not but all of the requests seem to have the header

x-aaaaaaaaaaaa: 1

so I've just blocked that with mod_rewrite
Fri, 04 Feb 2005 at 11:13:55 GMT Link

8. Ren said:

Wouldn't say it was completely useless, if the Referer contains a secret token in someway, then it becomes difficult to forge.
Fri, 04 Feb 2005 at 12:43:35 GMT Link

9. Ben Ramsey said:

Referer checking can provide just one more layer of security. It's not fool-proof, but it definitely protects against people creating an HTML form that uses your form and lives on another site. Still, anyone can create their own script to spoof your form by setting a referer header (PEAR::HTTP_Request makes it relatively easy to do this, as Harry pointed out). Nevertheless, checking the referer is better than nothing, but using your method, Chris, is probably the best route.

Mathieu, I don't see the advantage of gaining the $token through a GET request. The script that POSTs to process.php would still need $token set in their session, since process.php would do a check like:

if ($_SESSION['token'] == $_POST['token']) {

// process the form

}
Fri, 04 Feb 2005 at 15:25:02 GMT Link

10. Leendert Brouwer said:

As Mathieu said, it's pretty easy to still "spoof", or, formulated in a different way, "script requests against it". Using a script it'd go along the lines of this:

- curl -c mycookie.txt http://url.to/form.php

session cookie was set from there, and session id is now in mycookie.txt

- parse what that curl statement responded, get the value from the hidden field "token"

- curl -b cookie.txt -d "token=parsedtoken&foo=bar&bar=foo" http://malicious.script/still.php

been there, done that..
Fri, 04 Feb 2005 at 15:47:23 GMT Link

11. Chris Shiflett said:

HarryFuecks writes:

> Makes me wonder if this is a result of browsers being too good at hiding

> users from the nature of HTTP? Perhaps an effective remedy would be a hands

> on tutorial of writing an HTTP client with something like PEAR::HTTP_Request

> (or even from scratch with fsockopen).

That's a good point. I think many problems stem from developers not understanding the origin of data. In this case, I think it also stems from developers not understanding how forms work. I have spoken with many people who think that the request is sent by the site that is hosting the form. So, perhaps some developers believe that it is PHP that is providing Referer, making it trustworthy.

My concern is that there are plenty who know that it can be forged and still think it offers some protection. This is what confuses me, because everyone who knows the slightest thing about the web knows exactly what this value should be. It would offer more protection to make an attacker choose heads or tails. At least then the attacker will only guess it correctly about half the time instead of every time.

> Great sentence ;)

Desperation breeds humor. :-)

ryan king writes:

> I certainly agree that referer checking is by no means sufficient for

> security, But, still, my code's all going to have referer checking in it

> because I'll take any little bit of help I can get.

But that doesn't help. Because an attacker can choose the correct Referer to send every single time, this approach never prevents an attack. As I just mentioned, it would be better to make an attacker choose heads or tails.

Mathieu writes:

> Chris, you can easily write a script that'll first retrieve the page through

> GET => filter out the token => use that in a POST along with data you want to

> muck around with.

This is what I thought you meant. If a user knowingly submits a request to your site that abides by your rules, then it is not being spoofed.

This technique is not a substitute for data filtering. In fact, nothing is. Always filter your data.

Alan Knowles writes:

> I decided to actually implement the javascript form mods today to try and

> reduce blog spamming. While not a perfect answer to this, it adds a level of

> customization to deal with the form autoposting that is so specific to a

> particular site that it may not be viable for a blog spammer to implement.

It sounds like you're talking about automating a form, which I consider to be a separate topic.

Ren writes:

> Wouldn't say it was completely useless, if the Referer contains a secret

> token in someway, then it becomes difficult to forge.

Referer contains the previous URL visited. It is not a secret token.

In fact, Referer checking is the opposite of what you want. It offers you no protection and can interfere with your legitimate users (some proxies and third-party software strip or modify Referer). In other words, you're making things hard for the good guys and easy for the bad guys. Always try to make things hard for the bad guys and easy for the good guys.

Leendert Brouwer writes:

> been there, done that.

Yes, but what you describe is not form spoofing.

Form spoofing can be one of two things:

1. Cross-Site Request Forgery - The form abides by your rules, but the user sending it is a victim of CSRF and unaware. Therefore, the form is being spoofed by the attacker, and the user is providing an extra layer of diguise.

2. Manipulated Data - The user of your application is the attacker and sends unexpected data back to you in the form by spoofing the form or the raw HTTP request.

Using a secret token helps prevent the first case, and data filtering helps prevent the second. Referer checking helps prevent neither.
Fri, 04 Feb 2005 at 16:17:54 GMT Link

12. Chris said:

As I understand it referrer checking does help Cross Site Request Forgery when a malicious site for example submits a form in an iframe. Referrer checks prevent this because the remote site can not change the referrer for that user.
Fri, 04 Feb 2005 at 17:36:34 GMT Link

13. Ren said:

Ren writes:

> Wouldn't say it was completely useless, if the Referer contains a secret

> token in someway, then it becomes difficult to forge.

> Referer contains the previous URL > visited. It is not a secret token.

I didnt mean include it directly.

I just took dim view of the following

"To spoof this form, an attacker sends an HTTP POST request to http://example.org/process.php. Assuming the developer who wrote process.php is relying on Referer checking to prevent form spoofing, guess what the expected value is? Does this really seem like a big secret? An attacker will get this right every single time."

Which I dont think is accurate. As "Referer checking" is very vague.

If a $url is appended with extra parameter like

$url .= '&token='.MD5(SECRET.$url);

The "Referer checking" can then dissemble the referer and validate it actually computes to the expected token.

Whilst I dont claim this is suitable for all situations, or indeed any, I just think its important to know that "Referer checking" can work, and prevent an attacker spoofing. As they have to compute the correct hash, without the secret.

Fri, 04 Feb 2005 at 18:38:32 GMT Link

14. Louis-Philippe Huberdeau said:

No need to play with sockets to spoof a referer or any parameter of a request. PHP itself has built-in functionnalities for it such as stream context parameters or cURL if it gets real complex. I guess the best strategy is to log anomalies in the requests and be a little agressive towards them (temp blocking of IP?). Unless you do it all day long, it will take several times to forge a correct request to bypass security.

Bypassing security and simulating actual browser requests to access data can end up being quite entertaining. (But I only do it when I have the right to!)
Fri, 04 Feb 2005 at 22:28:27 GMT Link

15. Thomas Bley said:

Hi chris,

I think the example is a good start for beginners, but today the real bad guys use a lot more tricks ...

registering the remote-ip address (or the x-forwarded-for ...) in the session does a bit more. next thing is seperating good from bad html and replace the quotes with &xy; ...

bye

tom
Sat, 05 Feb 2005 at 21:16:42 GMT Link

16. Chris Shiflett said:

Chris writes:

> As I understand it referrer checking does help Cross Site Request Forgery

> when a malicious site for example submits a form in an iframe. Referrer

> checks prevent this because the remote site can not change the referrer for

> that user.

I hesitate to say you're right, but you are. :-)

However, this is still a poor practice, because it can prevent legitimate requests from legitimate users. A saefguard should never interfere with a legitimate user's use of your application, otherwise the "damage" is already done - your paranoia harms your application more than an attacker.

Louis-Philippe Huberdeau writes:

> No need to play with sockets to spoof a referer or any parameter of a

> request. PHP itself has built-in functionnalities for it such as stream

> context parameters or cURL if it gets real complex.

Here are some examples of sending a POST request with PHP:

http://shiflett.org/hacks/php/http_post

http://shiflett.org/hacks/php/streams_post

Thomas Bley writes:

> I think the example is a good start for beginners, but today the real bad

> guys use a lot more tricks ...

Blind paranoia is never a good security practice. Adhering to strong security practices and taking specific steps to help prevent the attacks you're aware of is much, much better.

Also, while I'm sure there are plenty of "tricks" that I'm not aware of, I think you underestimate me. :-)

> registering the remote-ip address (or the x-forwarded-for ...) in the session

> does a bit more. next thing is seperating good from bad html and replace the

> quotes with &xy; ...

There are two problems I see here.

The first is that relying on the IP address for identification is a terrible practice and not one that I would ever recommend. If you are developing an intranet site where your network topology is very simplistic and understood, it's not so terrible, but I still wouldn't recommend it.

The second problem is that you mention replacing quotes with an HTML entity. This has nothing at all to do with form spoofing. It is a technique you should use to escape data before output, although you should escape all HTML entities, not just quotes. The reason I see this as a problem is that I fear that you think these two practices address most security concerns, and they do not. For a more thorough (but still very incomplete) coverage of some security concerns, please refer to the PHP Security Guide:

http://phpsec.org/projects/guide/
Sat, 05 Feb 2005 at 21:35:21 GMT Link

17. Chris said:

> I hesitate to say you're right, but > you are. :-)

>

> However, this is still a poor practice, because it can

> prevent legitimate requests from legitimate users.

> A saefguard should never interfere with a legitimate user's use of your application,

> otherwise the "damage" is already done - your paranoia harms your

> application more than an attacker.

Sort of. If a browser does not set a referrer then you don't check for it. This then acts as added security for browsers (and thus people) that do send the referrer. A remote site can't prevent a site from sending the referrer, so if you set a referrer normally and a site uses a referrer check then you are now imune from CSRF post calls.

For those people that do not set referrer you have to use another security check. You can either use a token (although these can sometimes be leaked) or a turing test.

In my mind to prevent CSRF my policy would be:

a) token for all users

b) referrer check for all users that have it enabled

c) turing test (password / gotcha) for high level checks

d) turing test + email confirmation for the very highest level checks (change password / email address).
Mon, 07 Feb 2005 at 03:40:00 GMT Link

18. Chris Shiflett said:

> Sort of. If a browser does not set a referrer then you don't check for it.

You're forgetting those users who have either installed software to strip Referer or go through an HTTP proxy (voluntarily or involuntarily) that does so. Despite claiming to strip Referer, this software almost always uses the opportunity to promote itself:

Referer: Blocked by SecureBlock (secureblock.com)

The rest of your list is pretty sound.
Mon, 07 Feb 2005 at 04:07:54 GMT Link

19. Chris said:

I guess you just check for a valid referrer when the user registers and save that as a setting. Also, if someone fails a check you can just give them a gotcha to enable them to turn of referrer checking on their account; thats a one time thing - added security for 99% of people, a single gotcaha extra check for the 1% who don't use it.

Also, I think some widespread software does use a referrer check (vBulletin) for example I believe.
Mon, 07 Feb 2005 at 07:06:33 GMT Link

20. Tim Traver said:

Chris, funny how so many people have read about it but still come back with "yeah, but at least its a little security."

I was laughing at some of the posts. They just don't get that the referrer is one of the headers that anyone can send through a script or by hand for that matter...

Its like putting the key under the mat...
Fri, 18 Feb 2005 at 06:14:30 GMT Link

21. Chris Shiflett said:

I'd better find a better hiding spot for my key. :-)

Actually, Tim, Chris has a valid point about Referer. For attacks like CSRF (see http://shiflett.org/articles/foiling-cross-site-attacks for more information on that) where the request is actually coming from the victim and not the attacker, Referer can be useful. The challenge in this case is making sure not to adversely affect legitimate users. I'm not convinced that it's worth it, but it's an interesting concept (and hard to explain without people misinterpreting and thinking that Referer helps to prevent form spoofing). We're discussing it on the OWASP lists right now, and something might appear in the next revision of the OWASP Guide.
Fri, 18 Feb 2005 at 06:19:46 GMT Link

22. diyaudio guy said:

Refrerers are really useless. Even those who think they'll help just a tiny bit should actually reconsider their position. The check is not worth not even 0.1 usec

The way to go is some sort of token(fingerprint) inside of a hidden field and another fingerprint for proper session identification. Just don't store the token inside $_SESSION['token'] for all forms... What happens when the "good" guy opens up multiple windows with forms...?
Wed, 16 Mar 2005 at 13:25:03 GMT Link

23. Nikolas Coukouma said:

Generating user-specific secrets for each form is indeed the right way to go about this. As diyaudio points out, users should be able to have multiple secrets simultaneously, so session and/or cookies are not a good way to go about it. So, you just need to add a secrets table (columns: user, secret).

I will also note that Gecko-based browsers will not send a Referer header if it's SSL/TLS->unsecure (e.g. GET http://foo.com/ Referer: https://bar.com). That's a guaranteed bypass for 5-10% of users.
Mon, 20 Feb 2006 at 10:55:27 GMT Link

Amir wrote:: Hi chris! Please check this and guide me: http://forums.devnetwork.net/viewtopic.php?f=34&t=8...; Posted in
Nathan Bentley wrote:: Hi Chris, A great tutorial, which should help a lot of people! We implemented something simil...; Posted in
Daniel S wrote:: Just recently I sold my 1.gen Macbook(core duo version). And to be honest, I don't miss it for on...; Posted in Top X List of Mac OS X Annoyances
Buke Beyond wrote:: I agree it is ridiculous that php is doing this. I am using php for generating commands for othe...; Posted in PHP Stripping Newlines
Davis Ford wrote:: I agree, although I have a list of many more annoyances. However, rather than complain about the...; Posted in Top X List of Mac OS X Annoyances