REST based message queue

I’ve been playing with lots of message queues lately, and with each of one, a protocol is favoured. There are modules for all protocols for a given MQ if you look further, but usually only one of them is more developed than others. That’s ok, because it’s hard to find willing people to help in open source projects.

Based on what I saw and used, a simple queue would suffice for most of cases. I tried Amazon SQS and the idea behind it is nice and simple. As I was looking to build a small “cloud-y” infra structure for my projects, I’ve started to build such queue using Google’s AppEngine. This was almost 1 year ago and just last week I found the original code, which immediatly I saw to clean, add API Keys and make it public.

A quick write-up on python decorators and memcached

…or how to do proper caching without heavily modifing your code.

I’ve been using this technique on django views and Juno for some time. It’s just a decorator which inspect the parameters given to a function to search for a key on memcached.

You will find two files on github: no_decorator_example.py and decorator_example_memcache.py. Both of them search on twitter for a given term. The first script executes the search everytime, and the second script implements a 60 second cache, which uses the parameter (search term) as key.

I’m sure there must be another way to deal with parameters without the wrapper class, but so far nothing I’ve tried gave the same result.

For  Django views, one could go so far as using return HttpResponse(your_response, mimetype=’text_plain’) too.

Enjoy.

Github repo

I’m uploading almost all code from this blog to my repo at github: http://github.com/gleicon/zenmachine .

There’s some really ugly stuff, old ruby constructs, but all in all it’s been fun to review and update them. One of my top posts of all time, about text classification and ruby is getting a review and will be uploaded too.

Any code you want ? Questions ? Feel free to post them in comments.

 

Twisted COMET

I was about to write a post about NGINX, Python, Twisted and COMET, but it got so long that I decided to break it in 2 or 3 parts.

The first one is a kind of follow-up to the last post . This time the subject is a script that will search for a given word in twitter, and update the results in a continuous fashion. Once you point your browser to http://localhost:8000/ it will start to receive the results from time to time, as a big download (which is what COMET is about).

 

twisted and comet (comet in 60 seconds)

I’ve been looking into twisted to build a comet based app. It’s not a hard task, given that you can tune a lot of parameters (including which kind of reactor), but the basics are very interesting. Combining this approach along with a nginx based architecture has given me excellent results.

More twitter: naive friend recommendation

To finish the holy trilogy of twitter madness, there goes a naive friend recommender script for twitter. It basically applies a simple idea over the social graph: from all my friends, select all their friends that I don’t follow, rank them by ocurrence, filter against who is already in my list and recommend them to me.

A twitter bot

One of the first experiments that I did with python and twitter was a BOT. I was interested in testing how would an interactive application work using twitter, and a reasonable model for me is that it would answer back when a user sent a message, after performing a given action.

Data mining – Harvesting information from twitter

The whole twitter’s user base provide us with a snapshot of what is hip and what is not right now. Using its trends and searching, it’s possible to harvest a lot of information. The heavy load of crawling and storing such huge volume is naturally handled by them.

An office docs to flash converter (i.e. my slideshare clone)

Since a friend of mine told me of using OpenOffice as a daemon to run tasks automatically, I thought that would be nice to try it as a part of a proof of concept to a slideshare mini clone. It would be a matter of uploading the original file, convert it using OO and displaying a page along with it. There`s an API and many clients. I choose not to develop a new client and used JodConverter. Of course I would have to develop a new converter if I wanted to inject or run customized procedures over a document.

My book at google books

Long ago I wrote a book about Linux Programming (portuguese only). I just saw that it is added to google books at http://books.google.com/books?id=nMoQKiQyHwwC . There are some cute previews, including a serial port circuit picture