About the Author

Chris Shiflett

Hi, I'm Chris, a web developer and a founding member of Analog. I live and work in Brooklyn, NY.


Auto Increment with MongoDB

We are currently working on an app that uses a number of technologies, including PHP, Python, and MongoDB. Recently, a need arose to use sequential identifiers for users, similar to an auto_increment column in MySQL.

If you've used MongoDB, you might be familiar with the default behavior of using a UUID as the primary key. This is convenient, especially if you partition your database across servers, because you don't have to coordinate the primary key in any way. If you use sequential identifiers (as I demonstrate in this post), you can use multiple servers and interleave identifiers by advancing each server's sequence by the total number of servers. (For example, with two servers, advance each sequence by two, so one server generates even identifiers, and the other generates odd.)

I'd rather not discuss the advantages and disadvantages of either approach, because it's exactly this debate that makes it very difficult to find any useful information on using sequential identifiers with MongoDB. Instead, I'm just going to explain how I did it, and hope this is helpful to someone. :-)

First, create a sequence collection that you can use to determine the next identifier in the sequence. The following creates a collection called seq that has a single sequence in it (for users), but you can add as many as you need:

db.seq.insert({"_id":"users", "seq":new NumberLong(1)});

If you assign seq to 1 instead of new NumberLong(1), it will be interpreted as a float due to a JavaScript quirk.

Before adding a new user, you need to increment the sequence by one and fetch the next identifier. Fortunately, the findandmodify() command provides an atomic way to do this. Using the MongoDB shell, the command would look something like this:

db.seq.findAndModify({
    query: {"_id":"users"},
    update: {$inc: {"seq":1}},
    new: true
});

Because I'm using Lithium, I added a method for fetching the next identifier to my User model:

<?php
 
namespace app\models;
 
class User extends \lithium\data\Model {
 
    static public function seq() {
        $seq = static::_connection()->connection->command(
            array('findandmodify' => 'seq',
                  'query' => array('_id' => 'users'),
                  'update' => array('$inc' => array('seq' => 1)),
                  'new' => TRUE
            )
        );
 
        return $seq['value']['seq'];
    }
 
}
 
?>

If you're not using Lithium, you can use the MongoDB class to execute a command().

With this in place, adding a new user is a simple process. I create an array called $data with everything I want to store for a user, and then do the following:

<?php
 
$user = User::create($data);
$user->_id = User::seq();
$success = $user->save();
 
?>

This example should be easy to adapt to any environment. Once you have the next identifier in the sequence, you simply store it as you would any other data.

I hope to blog more about both MongoDB and Lithium. As these technologies are still pretty new to me, please feel free to point out any improvements. I'll update the post accordingly.

About This Post

Auto Increment with MongoDB was posted on Thu, 29 Jul 2010 at 19:52:18 GMT.

8 Comments

1. Ivo's GravatarIvo said:

Although you did mention that you werent going to discuss the why, I can't think of a single valid reason to force ids to be sequential, so out of curiosity I hope you can give an example.

Thu, 29 Jul 2010 at 21:27:50 GMT Link


2. Chris Shiflett's GravatarChris Shiflett said:

Hey Ivo,

Andrei is best suited to give a full response, since he's the one who researched this before deciding on sequential identifiers.

This post from Kellan explains why Flickr chose sequential identifiers over UUIDs. It's about MySQL rather than MongoDB, but the background and reasoning is very similar. It's a good start.

Thu, 29 Jul 2010 at 22:01:04 GMT Link


3. John Judy's GravatarJohn Judy said:

How do you avoid race conditions with this? Once you get to a certain traffic volume two or more people are bound to make this request at the same time.

Fri, 30 Jul 2010 at 02:16:16 GMT Link


4. Chris Shiflett's GravatarChris Shiflett said:

Hi John,

How do you avoid race conditions with this?

The findandmodify() command is atomic, so there is no race condition.

Fri, 30 Jul 2010 at 02:18:51 GMT Link


5. Cesar Rodas's GravatarCesar Rodas said:

Hi Chris,

Thanks for this terrific post. Few month ago I was trying to figure out how to get autoincrement in a safe way with another guy on IRC, I never thought about findAndModify!

I'm adding Autoincrement support to my MongoDB class -- http://github.com/crodas/ActiveMongo

Regards,

Sat, 31 Jul 2010 at 21:15:12 GMT Link


6. Chris Shiflett's GravatarChris Shiflett said:

Glad this was helpful, Cesar. :-)

Sun, 01 Aug 2010 at 22:06:20 GMT Link


7. Mike Girouard's GravatarMike Girouard said:

Well done, as always. I look forward to seeing more Mongo and Lithium posts. Not sure if you were there, but Nate slipped a few Lithium references through in the last NYPHP meeting. I'm pretty interested in seeing this.

Getting back on topic, I use this exact technique with a lot of my apps; though I can't say I've ever had the need to use it as an _id. For example, when I converted my internal financial management app to Mongo, I use increment collections for bumping up invoice numbers, project numbers, etc.

Tue, 10 Aug 2010 at 23:05:28 GMT Link


8. Fabio Silva's GravatarFabio Silva said:

I don't know about MongoDB, but using identity primary key (sequence numbers) is a waste of space.

Some would argue that ids are faster, because of their data type (indexing int or mediumint would be faster than indexing the first characters of a char/varchar type). But, in my opinion, it's better to use a natural key than those artificial ones.

offtopic: My name have a "´" on letter "a". I couldn't post because of that. Also, I had to type my comment again because the form implemented here lost my previous information.

Mon, 16 Aug 2010 at 13:53:02 GMT Link


Post A Comment

Personal Details and Comment

Style Guide

Line breaks are converted to paragraphs. Also use:

  • <a href="" title="">text</a>1
  • <em>text</em>
  • <blockquote><p>text</p></blockquote>
  • <code>2  <?php  if ($foo) {      $foo = TRUE;  }  ?></code>
  1. Note: <code> can be used inline (e.g. in paragraphs) or in a block as shown. Include whitespace and newlines in blocks.

Please enter Chris (my first name) below. This is a primitive spam prevention technique, and I apologize for the inconvenience.

Preview and Submit

Upcoming Events

Brooklyn Beta

21 - 22 Oct 2010

At The Invisible Dog, Brooklyn, New York.

New Comments

Mario Arroyo wrote:

The article is really very good and the users comments and external links to another articles jus...

Posted in
Raphael Almeida wrote:

I realy like hiphop music, but this is very crazy! We'll use it in user group PHP conference at ...

Posted in PHP Anthem
Mal wrote:

Having used smarty for many years, this has never been a problem for me, but after building a web...

Posted in PHP Stripping Newlines
Satya wrote:

Thanks for the info. I have posted the news here on my page: http://www.facebook.com/pages/Web-Sc...

Posted in PHP Anthem
John wrote:

Oh, you need to press "save your password".

Posted in Mozilla Account Manager

Browse Comments


Work and Books

Analog Essential PHP Security HTTP Developer's Handbook