Sounds cocky but that’s it. Using RestMQ, which builds on twisted, cyclone and a stack of well proven software, you can provide your applications with a robust and flexible queue over http protocol. It already was possible using COMET and GET/POST/DELETE requests, but now with websockets support it got to a new level.

Release gibberish apart there is a cool small app which streams twitter data to a html/css/js based app. Why all that work, would you say, if I can use it directly (well, not that easy because you need to provide username/pass to twitter streaming service) ? Well, for mashing up data and filtering it before delivery. Also, twitter is a convenient source of streaming data for tests, but usually you would roll up your own data source.

An actor based concurrency model or browser-based map/reduce is trivial, just like retrieving data, programming the proper javascript code to do all the work and post i back to a new queue.

Check the repository at and a live demo at Note that it needs a websockets capable browser (chrome or a recent webkit build) and it may be down due to my bandwidth constraints 🙂

Node.js and COMET

January 31, 2010

Today I started to study node.js, for these last days there was a lot of interesting posts about it.

So to begin understanding it, I setup to port two COMET based examples I did back when I started using twisted to do it.

Note that this may contain javascript misconceptions, as I’m not experienced with it and my js idiom is more geared towards python than proper js.

The first example is a basic comet server which prints a string to each connected user

simple comet

The other example sends the result of twitter searchs to all connected users:

twitter and comet

Node.JS presents a different toolset to solve the current problems we have. It used the excellent libev, among V8 and other cool stuff, and JS is a very modern and flexible language (at least more than I thought about it when trying to do web stuff).


To finish the holy trilogy of twitter madness, there goes a naive friend recommender script for twitter. It basically applies a simple idea over the social graph: from all my friends, select all their friends that I don’t follow, rank them by ocurrence, filter against who is already in my list and recommend them to me. Read the rest of this entry »

A twitter bot

August 21, 2009

One of the first experiments that I did with python and twitter was a BOT. I was interested in testing how would an interactive application work using twitter, and a reasonable model for me is that it would answer back when a user sent a message, after performing a given action. Read the rest of this entry »

The whole twitter’s user base provide us with a snapshot of what is hip and what is not right now. Using its trends and searching, it’s possible to harvest a lot of information. The heavy load of crawling and storing such huge volume is naturally handled by them.

Read the rest of this entry »