MongoDB and Redis for an URL shortener

March 1, 2010

I’ve wanted to prototype an URL shortener some time ago to have a measure on how long it would take to do using a NoSQL DB to store stats and do analytics. The main reason would do it using cyclone and MongoDB, and later build over it to create a tracker, web analytics tool and maybe a NGINX module.

As there are many topics discussing to which use each new NoSQL db belongs too, I thought that it may be natural to use more than one DB to specific tasks.

I setup to do it this weekend. From MongoDB I got the ease to create, change, manage documents and collections. It’s really refreshing start using and changing documents without worrying about migrations, schemas, tables and so on. From the initial idea to the final document, all I had to do was fire-up a python prompt (lovely BPython) and mess around with my documents. I soon found out that while MongoDB would be killer for stats storage, I missed Redis pragmatism regarding atomic operations. You can setup a counter with less code and worries that would need in a MongoDB only environment.

So I set up to implement a Redis unique global counter (using INCR) and fast encoded url to url cache (for redirects). Naturally I got a encoded url to mongo ObjectId index which is very fast. Also, I could cook up a nginx module which queries Redis to translate the URL, using its logs and a simple job script to gather stats and persist on MongoDB. But thats another project.

To real use there are some steps to be taken, but I consider it an excellent proof of concept for a weekend hack !

Enjoy it. The repository is at http://github.com/gleicon/uurl

About these ads

4 Responses to “MongoDB and Redis for an URL shortener”

  1. ehwizard Says:

    As for mongodb tracking, its as simple as
    db.stats.update( { _id : xxx } , { $inc : { clicks : 1 } } , true );

    Some more examples:

    http://github.com/mongodb/mongo-snippets/blob/master/analytics.py

  2. gm Says:

    Hi,
    I already did that. The counter I mentioned is a unique id for URL hash generation (in a distributed setup I would need to ensure that I had unique ids for each url).
    Thanks


  3. […] MongoDB and Redis for an URL shortener […]


  4. […] Last week, I have mentioned a post by Gleicon Moraes on building an URL shortener using MongoDB and alternatively Redis. (Fri, 14 May 2010 04:08:00) [936] […]


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: