Roundtripping the HTTP Flowchart

It has long bugged many of the Webmachine hackers that this relationship with Alan Dean’s HTTP flowchart is one-way. Webmachine was made from that graph, but that graph wasn’t made from Webmachine. I decided to change that in my evenings last week.

Webmachine hackers are familiar with a certain flowchart representing the decisions made during the processing of an HTTP request. Webmachine was designed as a practical executable form of that flowchart.

It has long bugged many of the Webmachine hackers that this relationship is one-way, though. Webmachine was made from the graph, but the graph wasn’t made from Webmachine. I decided to change that in my evenings last week, while trying to take my mind off of Riak 1.0 testing.

This is a version of the HTTP flowchart that only a Webmachine hacker could love. It’s ugly and missing some information, but the important part is that it’s generated by parsing webmachine_decision_core.erl.

I’ve shared the code for generating this image in the gen-graph branch of my webmachine fork. Make sure you have Graphviz installed, then checkout that branch and run make graph && open docs/wdc_graph.png.

In addition to the PNG, you’ll also find a docs/wdc_graph.dot if you prefer to render to some other format.

If you’d really like to dig in, I suggest firing up an Erlang node and looking at the output of wdc_graph:parse("src/webmachine_decision_core.erl"):

[{v3b13, [ping],                     [v3b13b,503]},
 {v3b13b,[service_available],        [v3b12,503]},
 {v3b12, [known_methods],            [v3b11,501]},
 {v3b11, [uri_too_long],             [414,v3b10]},
 {v3b10, [allowed_methods,'RESPOND'],[v3b9,405]},
 {v3b9,  [malformed_request],        [400,v3b8]},
...

If you’ve looked through webmachine_decision_core at all, I think you’ll recognize what’s presented above: a list of tuples, each one representing the decision named by the first element, with the calls made to a resource module as the second element, and the possible outcomes as the third element. Call wdc_graph:dot/2 to convert those tuples to a DOT file.

There are a few holes in the generation. Some response codes are reached by decisions spread across the graph, causing long arrows to cross confusingly. The edges between decisions aren’t labeled with the criteria for following them. Some resource calls are left out (like those made from webmachine_decision_core:respond/1 and the response body producers and encoders). It’s good to have a nice list for future tinkering.

Baseball + Riak Map/Reduce: the Movie

If you have been following my posts about using Riak’s map/reduce system to compute baseball statistics via the luwak_mr library, or if they’re still sitting in your ‘read later’ pile, you may be interested in a presentation I gave yesterday on the same topic.

Video of that presentation is available on Vimeo. It covers most of the content in the blog posts, while also providing a little extra background about why luwak_mr was necessary for the work.

Map/reducing Luwak

I was inspired, this weekend, by off-list discussion of Luwak and by Guy Steele’s talk How to Think about Parallel Programming—Not!. The two seemed naturally attracted, and thus I created the luwak_mr module.

The luwak_mr module exposes simple a function that knows how to walk a Luwak file tree, and send the keys for each of its leaf nodes off to a Riak map/reduce process. This enables one to run a map function against each block in a luwak file. For example, one might split a large Latin-1 file into “words” (the luwak_mr_words module in the project is an example implementation of the method that Guy Steele presented).

And, yes, this blog has been dormant for a while … I’ve been busy. Lots of woodworking and travel. Making music has also begun to require more time, and yesterday I learned how to ski cross-country. Always busy, the life of a hobbyist.

Update: luwak_mr has also been accepted to the Riak function contrib. So if you’re in the habit of browsing there, fetch the latest.

London Erlang User Group: Riak Introduction

You announce that you’re visiting a foreign city, and suddenly your schedule is full of things to do there. 😉

The latest addition to my schedule is the London Erlang User Group. I’ll be giving an introduction to Riak, and also discussing the advantages and disadvantages we’ve experienced by choosing to develop in Erlang/OTP. If you’re interested, go register and then attend the meeting on April 21.

Now if I can just get some time for pints worked into that schedule as well, I’ll be set. I’m very anxious for some bitter/porter/etc. in the land of CAMRA. 🙂

Riak Screencast with Ben Ahlan

Announcing a screencast demonstrating basic setup and usage of Riak. Ben Ahlan’s Video Code Chat hosts me as guest, and we dive right in with some Python examples.

If you’ve been thinking of trying Riak, but hadn’t yet gotten around to downloading and experimenting, there’s one more resource available this morning that might tip you over the edge. Ben Ahlan and I recorded a screencast demonstrating basic setup and usage of Riak. If you can stand watching two guys mumble over a console for 40 minutes, you may find a tip or two to make your experimentation go smoothly. 😉

I also hear that Martin Scholl gave an awesome talk at NoSQL Berlin. I can’t wait to see the video.