Map/reducing Luwak

I was inspired, this weekend, by off-list discussion of Luwak and by Guy Steele’s talk How to Think about Parallel Programming—Not!. The two seemed naturally attracted, and thus I created the luwak_mr module.

The luwak_mr module exposes simple a function that knows how to walk a Luwak file tree, and send the keys for each of its leaf nodes off to a Riak map/reduce process. This enables one to run a map function against each block in a luwak file. For example, one might split a large Latin-1 file into “words” (the luwak_mr_words module in the project is an example implementation of the method that Guy Steele presented).

And, yes, this blog has been dormant for a while … I’ve been busy. Lots of woodworking and travel. Making music has also begun to require more time, and yesterday I learned how to ski cross-country. Always busy, the life of a hobbyist.

Update: luwak_mr has also been accepted to the Riak function contrib. So if you’re in the habit of browsing there, fetch the latest.

Vector Clocks on The Basho Blog

Interested in Riak? Wondering about these “vector clock” things? Check out my latest post on The Basho Blog.

As penance for last night’s blathering about shiny new gadgets, I’ve written a post with much more interesting content for The Basho Blog. If you’re picking up Riak and curious about what these “vector clocks” are, and how to use them, have a look.

Webmachine Bloggy Goodness

Webmachine (and Erlang use in general) is picking up. If you’re looking for a few more people to follow in relation to webmachine, I have some suggestions.

A whole bunch of people (no half-bunches in these parts) are picking up Webmachine and writing about their experiences. If you’re looking for a few more people to follow:

Andy Gross (a coworker) is writing about file uploads. Actual webmachine details to follow in his next post – this first one is about the client side of the equation. Update: that second post is up, and includes an example of the new “streamed body” feature in webmachine 1.3.

Seth Falcon is covering the design and implementation of a URL-shortening service. Again, webmachine details to follow, but there’s some good talk about using Mnesia for storage in the current post.

Marc Worrell is laying down the outline of his new CMS, Zophrenic. This looks like a very ambitious project, and I’m anxious to see its release.

Update: More blogs from the comments

Daniel Kwiecinski just covered redirecting (3xx responses), and also recently showed off his own static-file-serving resource. He also tips his hat to wmtrace, so he gets brownie points. 😉

Great reads, but don’t take my word for it.</burton>

Like/Shrug/Dislike Gains Traction

Steven Frank agrees that 3-choice voting (like/shrug/dislike) is plenty. It’s great to see someone else suggest it. Maybe we’ll see it pop up in a few more places.

BeerRiot solves the specific problem Steven is grappling with (the sea of “average” ratings) in another way, as well: all BeerRiot ratings are personalized. You’re less likely to have to deal with a mess of vague ratings on BeerRiot because a social clustering algorithm is being used to sift the wheat from the chaff. BeerRiot can figure out which users’ votes are more important for your score, and uses them to give you a clearer picture. This means the score you see is much more likely to be in-line with your impression of the beer, not just the “average community rating.”

For those interested, here is my reasoning for keeping the ‘neutral’ vote around, in opposition to John Gruber’s suggestion.

WWW is back

Unfortunately, I only found out this morning that some of you probably thought I had given up on BeerRiot. A month or so ago, I started hosting another domain on the same server, and thought I’d set up Yaws to handle that properly. Unfortunately, I botched it bad enough that while still worked, didn’t. Sorry!

Anyway, things are fixed now, so if you like prepending “www.”, welcome back!

You may have also questioned the lack of new feature announcements of late. The reason behind this is a giant rewrite I’ve been working on all year. More details in a few weeks, but it’s almost an entire redesign that touches every level. At the same time, work has been crazy busy all year. I’m excited to share the new stuff with you all, though, so keep me in your RSS feed, so you don’t miss the announcement.


Woah – so much for February, eh?

In case people are wondering what’s up in BeerRiot land, here’s the skinny:

January and February were crazy months. I switched day jobs when I switched the calendar, and that meant a lot of extra planning in January, and a lot of extra concentration in February.

Unfortunately, March probably won’t be much different around here. My new day job has a big deadline on April 1, and we’re probably going to be at a dead run from now until then.

But, lucky for me, this doesn’t mean I have to give up Erlang – quite the opposite! In fact, my day job now has me writing Erlang all day long. 🙂 At some point in the future, I hope to be able to reveal where I’m working and what I’m working on, but that’s under wraps at the moment (it’s a new venture that doesn’t want the cat out of the bag just yet).

In the meantime, I do still log in to BeerRiot every night to keep things running. So, if you’re still looking for good beer recommendations, keep checking in.


Hi all. In case you’re wondering why BeerRiot has been down most of today – it’s a hardware problem. Something about faulty drives on the machine hosting my VPS, which keeps throwing things into read-only mode.

Hopefully they’ll get it fixed soon.

DevHouse Boston Wrapup

Just got back from DevHouse Boston. Thought I’d spill a few thoughts here for reference later.

The day was amazing. Where else would you expect to walk up to a random brick house in the middle of a city at 10 on a Sunday morning and expect to find ten 20-something guys hanging around discussing the finer points of locking in the Python VM? Yep, that’s the kind of day it was – lots of guys who are fairly knowledgable about a wide set of web tools, ready to sling some code at whatever comes their way.

I spent the day hacking with four other guys at an idea that started for me at a blog post I read last week, but mutated in many ways as the day progressed. The blog post I read lamented the lame way that RSS readers do nothing but provide a big list of links to follow (okay, maybe with a snipet or two, but still just a big list).

The idea that finally took us through the day was to use an external site’s popularity metrics (Digg for us, since that was easy), and layout news stories with summaries in a way reminiscent of newspapers. More popular stories got prime placement with larger headlines. Topics were grouped into distinct sections. We were only able to get to proof-of-concept stage, but everyone who saw the demo was interested in seeing it progress.

I’m not sure I’ll be the one doing the progressing, though. We did everything in Python/Django, and I’m just not sure I’m interested in delving deeper into it. Although, it would probably be a good springboard for doing so, since I already understand what the app is supposed to do. The code’s in a Google Project, but I’m going to make you search for it, because if you really want to try it out, the search is going to be the easiest thing you do. You think there’s time to write documentation at one of these things?! 😉

Here’s a screen cap to prove I’m not making things up:

NewsGraf Demo

Anyway, as for the rest of the projects, visualization was a key theme. And, even more specific, simplification of visualization was big. A calendar that does nothing but let you write five lines of text on each day (no categorization, contacts, etc.). A wiki edits tracker that doesn’t bother with the specifics of edits, but only the fact that edits happened, and that they probably related to edits that happened on “nearby” pages. A way to just send REST requests and view their results, without having to get into lots of code. And that’s only covering half or less of the projects that presented – the rest were amazing too.

I have to say, the day made me miss college. There’s nothing like the feeling you get when you and a small group of people focus on some strange goal for a few intense hours. In the “real world”, I’ve tended to let “real world” things (like grocery shopping, car maintenance, a day job, etc.) get in the way of that. It’s good to get it back once in a while.

Anyway, a big thanks to Betahouse for the space, the sponsors for the food, and my team mates for keeping things going. I wish DevHouse Boston many more successful years.

The Middle of Nowhere

I just wanted to warn all those interested that admining will likely be slower than usual this weekend. I have to make a trip to the middle of nowhere. The only internet connection I’m likely to have will be in my hotel room. Even there, I’m rather worried that the connection will be so awful that I won’t be able to handle anything but the simplest tasks.

Actually, the worst part may be that the nearest brewery is hours away.

On the upside, though, the long travel time is likely to be spent hacking at new features. (Provided I don’t find myself stuck without documentation, of course.) So, hopefully, you all can look forward to new things next week.