A new home

As much as wordpress has been really a good platform, zenmachine is not getting as much attention as my github repo for some time. Much of the things that I write of right now are so different that I’ve decided to put up a new blog at http://blog.7co.cc/ and stop writing on zemachine. I will try to keep the content up as much as I can but that’s it for this blog. xoxox

ffffuuuu conf recap

The majority of well stablished tech conferences here in Brazil are converging to be a big meeting of empty hands, with comercial and vendor speakers, entrepreneurship and rework based talks or agile method praising. Needless to say, tech subjects get very superficial and with little or no real world experience background.

This year was a fortunate one because new conferences started to happen, as NoSQL:br, QConSP and RubyconfBR. While there were the fair share of imaginary stuff as tests and agile speakers, they had a lot of technical goodness to spare. At these conferences we could find people with real world worries and experience. That’s way bigger than knowing new vendors or methodologies, I wanted to find people who had proven real world knowledge to share.

The last conference I took part this year was the coming up of a plan me and Renato Lucindo talked about.

We always talked about a conference to talk about things that don’t work as planned, real world experiences and the impact of the fallacy of a methodology in a large scale production environment. Every time one of us talked to friend about it, they god excited and asked to take part. The first plan for ffffuuuu conf was to take place at a room in Lucindo’s place as an informal BBQ with friends.

Last month it all took form and we got an endorsement from Caelum, a brazilian tech-head school and technology center. They let us use their auditorium and paid for mid-afternoon lunch and all day coffee. It got serious and we started an invitation list. The first round got more than 100 requests but we had only 50 places to fill. The website was up in less than a day with a list of speakers. After that we got overwhelmed by invitation requests.

The date was set to November 20, starting 9 am, with 6 speakers and 6 lightning talks. The conference website http://ffffuuuu.me/ have all slides and the videos will be uploaded as soon as they get edited.

I started with an experimental talk about Patterns of fail, which summed up a little bit of what I think it is our local issue. Followed Rodrigo Campos ‘Agile or Fragile’ talk, which was a real world report of sub optimal application of agile methodologies from an operations point of view. Along with ‘Itil for failers’ from Roberto Gaiser, it was a very candid view of system management and application development without the dream of ‘devops’. Things are not pretty.

Then we had a surprise presentation by Fabio Akita, who talked about association and superstition related to any methodology without the proper understanding of the expected results. Pause for lunch, we got back with Ivan Rocha’s ‘You shall not get excited’ talk, about new technologies applied to production environments. He got a case with Erlang and the walk changed to a class about distributed systems and Erlang internals. Fabio Trentini followed with ‘Hot not to use the right tool for the wrong reason’ , which went from awk and grep misuse to memcached patterns.

Closing the long talks, Renato Lucindo presented ‘Software Instability’, a very throughout talk about distributed systems, how not to test them, the fallacy of mocks and unit testing for systems which depends on remote procedure calls and are under heavy load.

The lightning talks started with Andrew de Andrade talking about management (Body Count as a metric for measuring managers), Pablo Borges with ‘The dynamics of ZOMG management’, Adolfo Sousa with ‘Como me desiludi com Agile’. Juliana Gaiba talked about ‘Ground Rules for User (un)Friendly’ and John D. Rowell closed with ‘How to CrAP’, an assessment about CAP and distributed databases.

So we covered real world cases for agile, development, tests, pair programming, UX, NoSQL, network, IT management, People management, Product Management, system architecture, system administration and Distributed systems in a single day. Everyone present took part in discussions and had the same feeling of discussing everything without believing in silver bullets. Better yet, we were talking with people who worked and made things work for at least 15 years down here in Brazil, people who worked for most of biggest ISPs and telecom hosts. This was not a group of evangelists or enthusiasts and the feeling at the end was that another conference was needed.

So, until 2012 where ffffuuuu conf might happen again. Thanks for everyone who made it possible.

RubyConfBR recap

Last week I presented at Rubyconf here in Brazil. It was the first edition named Rubyconf, but the folks behind it already had a tradition with Rails Summit. It was a huge event, specially considering that the main focus was a programming language. I got no exact numbers, but people where talking about more than 600 attendants. The site still up and you can see for yourself that there was a lot of interesting people talking.

I was invited by Fabio Akita to speak about non-blocking I/O. It was the first time here in Brazil that I got the opportunity to talk of such topics, and I’m glad that all conferences I attended or presented this year had more technical topics than vendor related and superficial concepts related to methodologies.

I had the opportunity to talk with Jim Weirich and Blaine Cook about what they are doing, and other folks that I happen to see once in a while, even living in Brazil.

Nando Vieira‘s lightning talk was one of the best I saw, along with Blaine Cook talking about webfinger and Emerson Macedo about Node.JS

There goes my slide deck

 

Publish Subscribe for collaborative text editing

It’s a big title, but the technology behind a simple collaborative editing textarea can be simple and interesting.

By using a derivative work from pubsub_ws along with diff match patch, it was possible to broadcast patches (yes, the same kind of patches you can create with diff -a) between peers using websockets and javascript.

The most interesting part is that I’m not a js expert, but I built the entire stack only using this language, from frontend to backend. There is also a patch to enable diff match patch to be used from node.js.

Here is the code

QConSP and NoSQL

Last sunday (Sep. 14) I presented along with Porcelli at QConSP. I never had presented to such a big audience before, but it was really fun. Besides architecture and data storage talk, we had a lotta fun.

I think that everytime I present something related to NoSQL, I end up talking about data distribution middleware. This time wasn’t different and I felt that by what people asked after the talk.

There goes our deck http://slidesha.re/9ieYNa and the wikipedia entry about Winnicot and the Transitional Object. Make sure to meet me again at RubyConf Brazil 2010. Thank you for coming !

Fun with Redis

I’ve been using Redis for projects on and off for some time, and there are some little hacks I’ve been doing and never extracted from bigger projects. Yesterday I had to sit home for some time doing a job that involved some idle time waiting, so it was time to hack.

First, I nailed a small RestMQ using Sinatra and Redis. Here’s the gist for the first version, which already works nice along with RestMQ. You can use it to expose a small part of your broker to the outside world. Later, talking with a colleague at work, I changed it a bit and ended up having the whole queue list and hard/soft get (deletes the message or just reads it). Another gist.

Then it was time of extracting Message Queue and Load Balancing patterns from code to my branch of the Redis Cookbook . Apart from the basic algorithm for RestMQ, there is a pattern which I sometimes use to do load balancing and replica spreading. It uses scored sets and although it seems naive, works pretty well along with consistent hashing.

After that I fixed some issues on RestMQ and txredisapi, the first were related to configuration issues and the later related to publish/subscribe.

About Pub/Sub, I ended up extracting a small PubSub server using Websockets, Redis and Node.js. It was initially embedded in another proxy I tried for RestMQ but it works well alone. Check the code. A little bit of code twisting and it can turn into a very flexible actor-based library for node.js. And of course, the PubSub thingy can also use redis as a presence server.

/me deserves pizza

websockets for python and twisted

RestMQ has a unique endpoint for consumers which uses websockets. As such, I implemented websockets for cyclone and twisted some time ago. Last July there was an upgrade to the protocol to implement a ‘secure’ handshake. This new spec broke most of the implementations because it mixed the upgrade headers part and the first 8 bytes from the content.

I’ve upgraded both cyclone and txwebsockets to understand the ‘old’ spec (hixie 75) and the new one. Basically the handshake involves extracting numbers from two headers (Sec-Websocket-Key1 and Sec-Websocket-Key2), dividing the resulting number by the number of spaces, concatenating them with 8 bytes read from the socket and sending back the md5 digest of this mess back after the new headers. Code to test and calculate the handshake from the headers value can be found here .

The rewrite ritual

Not that I am a fan of rewriting stuff, but there are some applications that I use to showcase technologies and from time to time I set up to rewrite them. As there is already a specification, its a matter of an exercise for new technologies. No over-specification and a very delimited time box.

Lately, two of these apps are a pastie clone and an url shortener. First I wrote them in ruby, them in python, python with twisted and now using Erlang and Mochiweb.

The code is on http://github.com/gleicon at uurl-erl and pasteme-erl repos, and as usual the html/css part is barely minimal. Also they are online at http://7co.cc and http://pasteme.7co.cc.

Both of them use Redis as a global increment and cache server, along with MongoDB for storing documents and stats. I find this to be a good match due to Redis atomic operations and pub/sub capabilities.

SQL AntiPatterns and noSQLbr

Last saturday I presented an updated version of what sent me the noSQL way at nosqlbr – the first noSQL conference in Brazil. Basically I talked about stuff I did and I saw when I gave consulting to legacy systems. Life is not pretty but there are ways out.

The event was great and I sent a recap to myNoSQL . I got really surprised that so many people got interested on noSQL and alternative/emergent technologies.

Websockets and RESTMQ – power to web applications

Sounds cocky but that’s it. Using RestMQ, which builds on twisted, cyclone and a stack of well proven software, you can provide your applications with a robust and flexible queue over http protocol. It already was possible using COMET and GET/POST/DELETE requests, but now with websockets support it got to a new level.

Release gibberish apart there is a cool small app which streams twitter data to a html/css/js based app. Why all that work, would you say, if I can use it directly (well, not that easy because you need to provide username/pass to twitter streaming service) ? Well, for mashing up data and filtering it before delivery. Also, twitter is a convenient source of streaming data for tests, but usually you would roll up your own data source.

An actor based concurrency model or browser-based map/reduce is trivial, just like retrieving data, programming the proper javascript code to do all the work and post i back to a new queue.

Check the repository at http://github.com/gleicon/restmq_websockets and a live demo at http://cowcat.net/restmq_ws/. Note that it needs a websockets capable browser (chrome or a recent webkit build) and it may be down due to my bandwidth constraints 🙂