About the Author

Chris Shiflett

Hi, I’m Chris: entrepreneur, community leader, husband, and father. I live and work in Boulder, CO.


The Accept Header

Years ago, I wrote a book on HTTP. At the time, there wasn't a simple, comprehensive reference for HTTP beyond the various specs, and a really good understanding of HTTP is important if you want to become a really good web developer, which I did. As I was reading and learning everything I could, I was surprised to learn that such a book didn't already exist.

Fast forward to today, and I can barely remember what I wrote. :-) One of the things I always struggle to remember is how to parse the Accept header. The spec has some examples you can use to refresh your memory, but I've never found RFCs to be a quick read, even when I know what I'm looking for.

Here's an example, the Accept header my browser sends:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json

It would be a little easier to read with some spaces:

Accept: text/html, application/xhtml+xml, application/xml;q=0.9, */*;q=0.8, application/json

Here's a quick reference for how to interpret the Accept header:

  • The q indicates quality, a measure of preference.
  • The default quality is 1.0, the highest preference.
  • Each q and the media type it references are delimited by a semicolon.

Here's the breakdown of this particular example:

text/html
It's common for the most preferred type to be listed first, with no quality specified.
application/xhtml+xml
XHTML served as application/xhtml+xmlis the next type that shares the highest preference. (This site serves XHTML as text/html. You can read more about this decision in the colophon.)
application/xml;q=0.9
XML has a quality of 0.9, indicating a high preference, but not as preferred as HTML and XHTML.
*/*;q=0.8
Asterisks are wildcards, so */* matches any type. This is similar to saying anything is fine, but the other types are preferred, because this is given the lowest quality.
application/json
I use JSONovich, which explicitly adds application/json as a supported media type.

It's the support for application/json that prompted me to blog about this. I've noticed a recent trend that affects sites with APIs when JSON is one of the supported formats. The trend is best shown by example. Here's what I see when I visit Netflix:

Netflix fails

Just a few days ago, I would see a web site. Now I see JSON. Pretty crazy, right?

Netflix isn't the first site to make this mistake; it's just the most recent. I experimented a bit to see if I could figure out what they were doing wrong, and I was disappointed with what I found. Would lowering the quality of application/json all the way to 0.0 make a difference?

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json;q=0.0

Nope, still JSON. The mere presence of the string "json" anywhere in the Accept header seems to be enough to cause the problem. The Accept header might be hard to read and parse, but they're not even trying.

I'd love to know why this is becoming more and more common. Is there a broken content negotiation example somewhere that everyone is using? I'm really curious.

I've reported the problem to Netflix via phone and Twitter. Hopefully they'll fix it soon.

About this post

The Accept Header was posted on Tue, 31 May 2011. If you liked it, follow me on Twitter or share:

16 comments

1.Jeremy Cook said:

Thanks for the informative post. I'm using Firefox 4 on Windows and I just installed JSONovich to try out this problem. I can successfully go to Netflix without any issues. I checked in Firebug and application/json is being sent in the accept header. The big difference is that I'm not a member of Netflix and get redirected to the signup page. Could be that this is an issue only after logging in to Netflix?

Tue, 31 May 2011 at 15:31:15 GMT Link


2.Chris Shiflett said:

Yeah, you need to be logged in to see this particular failure.

Tue, 31 May 2011 at 15:35:32 GMT Link


3.Jeremy Cook said:

So you get to part with your cash before finding out it doesn't work properly (with this extension installed)? Sounds like a great business model! :)

Tue, 31 May 2011 at 15:38:59 GMT Link


4.Paul Reinheimer said:

All of those newlines look like whitespace code.

http://en.wikipedia.org/wiki/Whites...mming_language)

Tue, 31 May 2011 at 16:16:13 GMT Link


5.Lorna Mitchell said:

Actually I have a service that does exactly this (http://api.joind.in)! It's a bug; I have a nasty hack in to check the accept header but it is rather stupid. If the code doesn't find something it understands, then it just sends json (since that's *my* first choice). This serves as a timely reminder to sit down and write a decent parser for this, since I haven't come across one I can just use.

Wed, 01 Jun 2011 at 08:23:35 GMT Link


6.Clay Hinson said:

Lorna:

I've just recently added Accept header parsing to the FRAPI codebase; the method is on Github if you'd like to use it.

Wed, 01 Jun 2011 at 15:29:34 GMT Link


7.Wez Furlong said:

Just so happens that I'm parsing this stuff for a hobby project this weekend; I have the following code snippet for PHP 5.3 that populates an array in order of preference.

Feel free to take and adapt as appropriate.

<?php
 
  protected function parseAcceptHeader() {
 
    $hdr = $this->headers['Accept']; 
 
    $accept = array();
 
    foreach (preg_split('/\s*,\s*/', $hdr) as $i => $term) {
 
      $o = new \stdclass;
 
      $o->pos = $i;
 
      if (preg_match(",^(\S+)\s*;\s*(?:q|level)=([0-9\.]+),i", $term, $M)) {
 
        $o->type = $M[1];
 
        $o->q = (double)$M[2];
 
      } else {
 
        $o->type = $term;
 
        $o->q = 1;
 
      }
 
      $accept[] = $o;
 
    } 
 
    usort($accept, function ($a, $b) {
 
      /* first tier: highest q factor wins */
 
      $diff = $b->q - $a->q;
 
      if ($diff > 0) { 
 
        $diff = 1;
 
      } else if ($diff < 0) {
 
        $diff = -1;
 
      } else {
 
        /* tie-breaker: first listed item wins */
 
        $diff = $a->pos - $b->pos;
 
      }
 
      return $diff;
 
    });
 
    $this->accept = array();
 
    foreach ($accept as $a) {
 
      $this->accept[$a->type] = $a->type;
 
    } 
 
  }
 
?>

And accompanying unit test:

<?php
 
  function testAcceptHeader() {
 
    $c = new MVC\Controller('', '', array(), array('Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json'));
 
    $this->assertEquals($c->accept, array(
 
      "text/html" => "text/html",
 
      "application/xhtml+xml" => "application/xhtml+xml",
 
      "application/json" => "application/json",
 
      "application/xml" => "application/xml",
 
      "*/*" => "*/*"
 
    ));
 
  } 
 
?>

Sun, 05 Jun 2011 at 01:06:17 GMT Link


8.Wez Furlong said:

Nice double-line spacing there :-/

Sun, 05 Jun 2011 at 01:06:57 GMT Link


9.Chris Shiflett said:

That's a bug. I'll fix it. :-)

Sun, 05 Jun 2011 at 02:54:49 GMT Link


10.Artem Nezvigin said:

Seems to me that this is a simple case of doing it wrong. It's interesting that few people spend the time to understand how the Accept header needs to be handled.

Actually this touches on a broader facet of web development that I find interesting. The barrier to entry is so low for writing a web applications that people can produce serious work without the requisite understanding of the technologies they are using.

This isn't a shot at Netflix. I think we're all guilty of this behavior. The web environment is so forgiving that we can do something the wrong way and it'll just work. Until it doesn't :)

Mon, 06 Jun 2011 at 18:24:54 GMT Link


11.Daniel Fahlke said:

I use the addon JSONview, which has an option if i want to add json to my accept header.

And i saw this problem several times too.

The problem is, for normal noone sets q=0, this is only a theoretical value. And as it is hard to parse, why should i try, if noone use the complex abilities of it.

In the end, it is a recommendation anyway and nearly noone change this value.

So why happens this problem? Its because a Developer tries to be smart. He think he can differentiate a ajax from a normal request over this (which is wrong, X-Requested-With:XMLHttpRequest is for this purpose)

Fri, 08 Jul 2011 at 20:56:26 GMT Link


12.Chris Shiflett said:

It took a couple of months, but it looks like Netflix finally fixed the problem.

Fri, 09 Sep 2011 at 16:25:27 GMT Link


13.Philip Tellis said:

My guess is that this is how they differentiate between requests that come from XHR and those that come through a regular user initiated request. Their JavaScript/Flash probably uses the Accept header to specify that they want JSON rather than HTML. I'd done a talk about using the Accept header with Ajax requests back in 2005, and it's almost exactly the same technique, except that I recommended that the Accept header should be exactly equal to application/javascript+json (the content types for JavaScript and JSON hadn't been standardised yet)

Mon, 17 Oct 2011 at 14:37:40 GMT Link


14.Andreas Wiik said:

Thanks for the code Wez :-) If you want to make a RESTfull API you need to handle the Accept header correctly.

Mon, 24 Oct 2011 at 13:38:37 GMT Link


15.Ben Ramsey said:

Is there a broken content negotiation example somewhere that everyone is using?

I've been playing around with the mimeparse library and converting it to use Composer, as well as conform to PSR standards. See: https://github.com/ramsey/mimeparse (shameless plug)

After re-reading your post, I decided to give this a try, using your second Accept line (with the quality parameter on application/json):

<?php
 
$accept = 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json;q=0.0';
 
$quality = \Bitworking\Mimeparse::quality('application/json', $accept);

Oddly enough, $quality comes out as the value 1, rather than 0, as one would expect.

Bingo!

This library has been around for a while; has versions in Python, Ruby, Erlang, Java, JavaScript, and PHP; and I suspect it is either widely used or widely imitated. I went back to the HTTP spec to see if a value of 0 or 0.0 is considered invalid or undefined, and it's clearly not (from RFC 2616, section 3.9):

A weight is normalized to a real number in the range 0 through 1, where 0 is the minimum and 1 the maximum value. If a parameter has a quality value of 0, then content with this parameter is `not acceptable' for the client.

This is clearly a bug in the library, which is also in my version of the library right now, until I fix it (or get a pull request). :-)

Tue, 17 Jul 2012 at 01:21:36 GMT Link


Hello! What’s your name?

Want to comment? Please connect with Twitter to join the discussion.