Twisted is a powerful python networking framework used in many projects. It helps a lot when you need to hack up a server or start prototyping and reuse the same code later. Its main component is a Reactor, which is a pattern for high performance event based network data exchange. There are a lot of servers provides, one of them a web server.

I built a small REST-like(alto not full REST) web service to store small text messages, by a give hash key, almost as a networked dictionary. The book ‘Twisted network programming essentials‘ provides some examples, but I found that the simple explanation about requests and child (chapter 4), was enough to start me up on how to handle requests as a hierarchy of http://host/resource/parameter.

This server has 3 methods: /check, /train and /delete. It is intended as a frontend for text classification systems, so that’s why I’ve used ‘train’ instead of ‘insert’ as resource name.

webserver.py

from twisted.web import server, resource, http
import message

class RootResource(resource.Resource):
    def __init__(self, messageStore):
        self.messageStore = messageStore
        resource.Resource.__init__(self)
        self.putChild('check', CheckMessageHandler(self.messageStore))
        self.putChild('train', TrainHandler(self.messageStore))
        self.putChild('delete', RemoveMessageHandler(self.messageStore))                      

    def getChild(self, path, request):
        return ShowMessage(self.messageStore, "")

class CheckMessageHandler(resource.Resource):
    def __init__(self, messageStore):
        self.messageStore = messageStore
        resource.Resource.__init__(self)

    def getChild(self, path, request):
        return ShowMessage(self.messageStore, path)

class TrainHandler(resource.Resource):
    def __init__(self, messageStore):
        self.messageStore = messageStore
        resource.Resource.__init__(self)

    def getChild(self, path, request):
        return EmptyChild(path)

    def render_GET(self, request):
	request.setResponseCode(http.NOT_FOUND)
	return """
		<html><body>use post method for direct insertion or form below<br>
		<form action='/train' method=POST>
		HashKey: <input type=text name=hashKey><br>
		<textarea name=body>Body</textarea><br>
		<input type=submit>
		</body></html>
	"""

    def render_POST(self, request):
	hashKey=request.args['hashKey'][0]
	body=request.args['body'][0]
	self.messageStore.setMessage(hashKey,body)
	return "Posted"

class RemoveMessageHandler(resource.Resource):
    def __init__(self, messageStore):
        self.messageStore = messageStore
        resource.Resource.__init__(self)

    def getChild(self, path, request):
        return DelMessage(self.messageStore, path)

class DelMessage(resource.Resource):
    def __init__(self, messageStore, path):
	self.path=path
        self.messageStore = messageStore
        resource.Resource.__init__(self)

    def getChild(self, path, request):
        return EmptyChild(path)

    def render_GET(self, request):
	if self.messageStore.hasMessage(self.path):
		self.messageStore.delMessage(self.path)
       		return """ msg %s deleted	""" % (self.path)
       	else:
		return """ msg not found for hashKey: %s""" % self.path

class EmptyChild(resource.Resource):
    def __init__(self, path):
        self.path = path
        resource.Resource.__init__(self)

    def render_GET(self, request):
	return ""

    def render_POST(self, request):
	return ""

    def getChild(self, path, request):
        return EmptyChild(path)

class ShowMessage(resource.Resource):
    def __init__(self, messageStore, path):
        self.messageStore = messageStore
	self.path = path
        resource.Resource.__init__(self)

    def render_GET(self, request):
	if self.messageStore.hasMessage(self.path):
       		return """hashKey: %s\n body: %s\n
            	""" % (self.path, self.messageStore.getMessage(self.path))
       	else:
		return """msg not found for hashKey: %s""" % self.path

    def getChild(self, path, request):
        return EmptyChild(path)

if __name__ == "__main__":
    import sys
    from twisted.internet import reactor
    messageStore = message.MessageStore(sys.argv[1])
    reactor.listenTCP(8082, server.Site(RootResource(messageStore)))
    reactor.run()

message.py

import pickle, os, sys

class MessageStore(object):
 "class for managing messages in the form: md5hash:text"

 def __init__(self, filename):
 self.filename = filename
 if os.path.exists(filename):
 self.messages = pickle.load(file(filename, 'r+b'))
 else:
 self.messages = {}

 def save(self):
 pickle.dump(self.messages, file(self.filename, 'w+b'))

 def hasMessage(self, hashKey):
 return self.messages.has_key(hashKey)

 def listMessageKeys(self):
 return self.messages.keys()

 def listMessages(self):
 return self.messages.items()

 def getMessage(self, hashKey):
 return self.messages[hashKey]

 def setMessage(self, hashKey, content):
 self.messages[hashKey] = content
 self.save()

 def delMessage(self, hashKey):
 del(self.messages[hashKey])

if __name__ == "__main__":
 messages = MessageStore(sys.argv[1])
 messages.setMessage('AAA', "oieee")
 print messages.listMessages()
 print messages.hasMessage('AAA')
 print messages.getMessage('AAA')
 print ["%s=%s" % (h, m) for h,m in messages.listMessages()]

The message.py module just implements a storage layer for the messages, using pickle as its main engine. The full server and its classes are on the file webserver.py, which takes care of all requests and redirections. Run it with $ python webserver.py data.dat.

To simulate an automatic post use curl -d “hashKey=key1;body=fulltextbodynoencoding” http://localhost:8082/train . There is a html form if you request the address http://localhost/train in your browser.

If you ran python message.py data.dat before starting webserver.py, you may already have a message in the datafile. To check it, go to http://localhost:8082/check/AAA. To delete it, go to http://localhost:8082/delete/AAA. You may do it using curl from the command line too.

The whole REST engine would have to support DELETE and PUT methods, and also, it would be nice to deliver a XML or JSON response for each method, but it would cloud out the beauty of twisted.

There are other ways to do what I did with EmptyChild class, but I wanted a recursive placeholder to stop python from raise unwanted exceptions.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: