PyWeek 6 has been announced!

Looks like they are getting ready for the next PyWeek competition. If you haven't heard about PyWeek yet, it is an informal competition where a lot of people create a game from scratch, in Python, in a week, based on a theme. I've been wanting to participate in one for a while. Hopefully my schedule will allow for it to happen this time around.

Tell Python to Shove it!

I recently stumbled across the shove python module when reading the excellent PyMOTW posting about the shelve module. The shove module is very similar to shelve, except that it provides multiple storage back ends (including some very interesting ones like S3 and SVN), a caching layer, and optional compression.

The very simplest usage of shove would look something like:

from shove import Shove
data = Shove()

This will give you a simple in-memory shove object that you can use just as you would a plain dict.

To use one of the other storage back ends, you pass a URI when creating the object. Some example are:

# Store persistent objects in files
file_data = Shove('file://path/to/file')     

# Store persistent objects in a BSDDB database file
bsdb_data = Shove('bsddb://path/to/file')   

# Store persistent objects in SVN                          
svn_data = Shove('svn://user:password@http://svn.mydomain.com/path/to/svn') 

# Store persistent objects in S3
s3_data = Shove('s3://s3_key:s3_secret@bucket')                       

You can also store data in any DB that SQLAlchemy supports by sending a SQLAlchemy DB URI to Shove (Note that this does require having SQLAlchemy installed).

# Store persistent objects in SQLite using SQLAlchemy
sqlite_data = Shove('sqlite:///relative/path/to/database.txt')

To compress data that you store, pass compress=True as an argument

# Store compressed data in the 'data.db' BSDDB file
bsddb_data = Shove('bsddb://data.db', compress=True)

Coming full circle now, I am finding this to be very useful with Mulib. Shove gives me simple persistent RESTful storage with Mulib. Taking a simple example from a previous post, it looks something like this:

from mulib import mu, stacked
from eventlet import api, httpd
from shove import Shove

# Create methods so that stacked knows how to handle Shove objects
stacked.add_consumer(Shove, stacked.consume_dict)
stacked.add_producer(Shove, stacked.produce_dict_html, 'text/html')
stacked.add_producer(Shove, stacked.produce_anything, '*/*')

root = Shove('bsddb://data.db')

# Initialize data if it is not there
root.setdefault('hello', 'Hello World')
root.setdefault('contacts', {})

httpd.server(
    api.tcp_listener(('0.0.0.0', 5000)),
    mu.SiteMap(root)
)

# Make sure to sync the data before we exit
root.sync()

It also wouldn't be that hard to add a greenlet that would sync the db periodically to help guard against loosing data if the server crashed. The one downside to this is that reading to or from the DB will block the app since it is not an async operation. I still need to figure out a decent way to handle that, but so far it hasn't been noticeable for the simple stuff that I am experimenting with at the moment.

I have not yet had time to explore the caching capabilities of Shove, but it also gives you several ways to cache the data (In memory, Memcached, etc.). It would also be interested in seeing a back end for Amazon's new SimpleDB service and also for git.

If you are looking for simple object persistence in Python, Shove is a very nice solution.

My first attempt at making sushi

I finally took the time to try to make my own maki sushi. After reading up on it, I was ready for total failure, but I think it turned out alright. Mmmmmmmmm...

A New Look

Update: I should never work on my blog template at 2:00am in the morning. It is funny how things can look so different the next day. :) Hopefully I can find some time soon to clean things up...

If you are reading this, you are probably noticing that things are just a little different. This is my first real stab at modifying a blogger template. I still have some work to do, but I think I am liking where it is going. Let me know what you guys think!

Medieval Helpdesk

There have been many times where I have had to provide support for computers and software in my life. While reading reddit.com, I came across this video that I can definitely relate to.

So what does the company you work for do again?

I often find friends asking the question about what Renesys does. It is usually a bit difficult to describe the main business that we are in. The recent events of the Middle East loosing internet access provides an opportunity to showcase the types of things that they work on. One of my co-workers, Jessica, has a great write up about it over here. Check it out if you are interested to find out more about some of the things we work on at Renesys.

Help! I'm Trapped in LEGO!

This weekend I found some libraries that makes it easy (or at least relatively easy) to render LEGOs in POV-Ray. If you are interested in doing something similar, check out:

www.antonraves.com/welcome.php
and
www.magnusviri.com/lego/downloads/povray_library.html

A Toy Chat Server with Eventlet and Mulib

Making Comet based chat servers seems to be quite the rage as of late. In my attempt at learning how to use Eventlet and Mulib, I thought that I would create my own little toy chat server. Full source (all 2 files of it :)) can be downloaded here. Note that you will need the latests SVN versions (or v0.2 if it has been released) of Eventlet and Mulib for this example to work.

Eventlet has a nice class for handling events called event (imagine that) in the eventlet.coros module. Calling the wait() function of an event will block the current code path until a send() is sent to the event, at which point all paths of code that are waiting for that event will receive the message sent. I use this mechanism to handle the long polling for new chat messages. The Mulib stacked module provides a nice interface that exposes events as a resource, so all you have to do is add your event object to your url heirarchy. the code looks something like this:

chat_listener = coros.event()
root = {'listen' : chat_listener}

This will expose the chat_listener at /listen.

We will also need to add a resource that receives new chat events from the web interface. Custom resources can be made by subclassing mulib.mu.Resource. In order to handle requests, override the appropriate handler(handle_get, handle_post, etc.). In this case we will override the handle_post function to look something like this:

class ChatMessage(mu.Resource):
    def handle_post(self, request):
        nick = request.get_arg('nick')
        message = request.get_arg('message')
        chat_listener.send("%s: %s" % (nick, message))
        chat_listener.reset()
        request.write('')

This will get the nick and message from the POST arguments and then send the message to chat_listener. Also notice that a call to chat_listener.reset() is called to reset the event, and allow future messages to be sent.

The last thing we need is some HTML and fancy JS to do the rest. Mulib provides a File resource (along with several other helper reousrces) in the mulib.resources module. So we expose our html file like this:

root = {
    'chat' : resources.File('chat.html'),
    'listen' : chat_listener,
    'message' : ChatMessage()
    }

This wil load the chat.html when a user browses /chat. Also noticed that we attached our ChatMessage resource to /message.

There isn't anything super fancy in the html file to really blog about. It is pretty much the standard AJAXy stuff you might see other places. I've included it in the source download so that you can check it out.

You can run the server from the source by running the chat.py file, and then browsing to http://localhost:5000/chat. Enter a message at the bottom and hit enter to see what happens!

I did run into a couple of problems with HTTP/1.1 support, so I had to force the server to send HTTP/1.0, until the bugs have been fixed. The Eventlet devs have been very helpful and responsive to my bug reports, so I hope to see them fixed in the near future. It also sounds like they have a lot planned for it in the future.

Next up... A full fledged MUD... Well maybe not quite yet, but I am getting there a step at a time :)