My Favorite Moment of 2013

It’s the last day of 2013, and I’m supposed to be finishing preparations for a cross-country move. But instead, I really want to recount my favorite moment of this past year.

On Friday, October 11, 2013, MIT’s Hobby Shop held a celebration to commemorate its 75th anniversary. The hobby shop is a place for the MIT community (students, faculty, alumni, and such) to … well, practice *manus* after stretching their *mens*. It’s a large room, filled with benches, power tools, and hand tools for working wood, metal, plastic, etc.

People use the Hobby Shop to build … things. Equipment for lab projects, musical instruments, furniture, signs, or whatever else they might dream. I was (sadly) not a member in college, but joined later to learn and use their large machinery when starting my bed.

The celebration in October included many member projects on display, one of which was a camera. Biyeun, its builder and user, gave a presentation about making and using her creation. In her introduction, she explained her discovery of view cameras and her instantaneous reaction: “I must build that.”

As I nodded my head in understanding of her sentiment, I saw heads all around the room do likewise. Building a machine gives you a different understanding of it that no variety of use ever will. Just a taste of such knowledge can cause everyday objects to practically scream at you forever afterward, “Imagine what it’s like to create me.” I knew that everyone nodding had heard that call.

The dean of student life, Chris Colombo, spoke as well. He was not a member of the Hobby Shop, but had good friends there. He expressed awe for the projects like Biyuen’s camera, that he had seen leave the shop, and a few minutes into his speech said something like, “I wish I knew how to build something like that.” As he took a breath afterward, I could just feel every shop member in the room struggle to restrain themselves from walking onto the stage, grabbing Chris by the elbow, and dragging him to the shop, to teach him how. “C’mon, I’ll show you,” were the words on every lip.

Realizing that I was surrounded by people that not only had wanted to know, and then spent time doing and learning, but now also wanted to show and teach, was my favorite moment in 2013. Finding people that are curious is not terribly hard. Finding those that will follow through on their curiosity can sometimes seem rare. But, finding one who actually wants to share what he or she has learned, by answering the endless naive questions of a beginner, is like winning the lottery. To be standing in a room full of such individuals was overwhelming.

Hobbies -= 1

I shut down a hobby today. BeerRiot, the site I started over six years ago, is now closed. I’m keeping the domain active, because I’ve used the name in other places, but browsers will see only a static archive of what used to be there.

BeerRiot began as an experiment. I wanted to learn about Erlang, and I needed a project to drive my curiosity. It worked, and I learned a good deal about modern web application development in the process. In fact, I learned enough about both that, through blogging about my progress, I was able to join up with a smart team and work in Erlang on web apps professionally.

In fact, even after the experiment paid off, BeerRiot remained my sandbox. New webservers, new storage techniques, new rendering processes, new API designs … I was able to practice with them all in a live setting before attempting to pull an entire team of engineers toward any of them.

So why would I give up my playground? Simply put: I don’t play there any more. My interests have moved on, and it’s time to remove the mental clutter of the service existing (no matter it’s reliability). Were the virtual server some physical object, I’d be putting it on a garage sale. As it is not, I will instead throw a tarball on a backup disk, and laugh when I find it in a few years.

What’s next? On the code side, more focus on that smart team and profession Erlang work I mentioned. On the hobby side … definitely not another web app. I’ll keep this blog up. No promises on changes to its post frequency, but readers will be among the first to know when I find a new thing.

Cheers.

I built a maple sleigh bed.

Some of you know that beyond beer and coding, I’m also an active woodworker. In this post, I’m excited to share the completion of my most recent project. Approximately 18 months after buying the first lumber, I’m now sleeping in in my hand-made hard maple sleigh bed.

The finished bed
The finished bed

It’s not my first piece of furniture, but it is, by far, the largest and most intricate to date. Despite some amateur imperfections (or are we calling those “artisanal qualities” these days?), I’m quite happy with how it turned out. It looks good, it sits straight, the matress fits, and it doesn’t squeak!

Slats hold the mattress
Slats hold the mattress

The matress is supported by fifteen poplar slats, each 4 inches wide by 3/4 inches thick, with an inch of space between them. That is to say, there’s no box spring. We bought the matress about a year ago, and we have used a futon frame (also a slatted frame), to support it since then. At “full” size, these slats seem to be fairly firm, but not hard. We like it.

Raw below, one coat of finish above
Raw below, one coat of finish above

The finish is a mixture of one part mineral oil to 4-5 parts beeswax, by volume. It’s not a hard, take-a-beating kind of finish, and it will need to be reapplied from time to time, but we couldn’t resist the beautiful natural color of the maple, the sweet honey smell, and the smooth matte texture. We don’t expect it to need to withstand much more than our touch and the seasonal humidity change anyway. Rumor also has it that the finish and wood should change color with exposure to the sun as the years go by, which will add a great living element to a long-loved piece of furniture.

Tenons on the footboard
Tenons on the footboard

You may notice that there is no metal hardware visible. The headboard and footboard are mortise-and-tenon boxes around a floating plywood panel. Glue and a good fit is all that’s holding them together.

Hidden bolt joinery
Hidden bolt joinery

Fear not, though, for I am not crazy enough to glue up a piece of furniture that cannot later be removed from a room. The side rails are attached to the headboard and footboard via bolts, but in a sneaky way. The bolts are recessed into the side rail from the inside. They protrude from the end of the side rail, and then pierce the headboard and footboard through a hole on each inner face. A square nut is captured in the tenon of the lower cross rail, to secure the end of the bolt. Wood pins also protrude from the end of the side rail, and slot into holes in the headboard and footboard, to prevent the side rail from spinning around the bolt.

Cleats, glued and screwed
Cleats, glued and screwed

Beyond connecting the headboard and footboard to each other, the side rail also holds the cleats, which support the slats for the matress. The cleats are attached with good, old-fashions glue-and-screws construction, to ensure they never pry themselves off.

Square Octagonal Sixteen sides Nearly round Polished
Progressively rounder

The crest rails atop the headboard and footboard are defining features of the bed. The first question I’m always asked about them is, “Did you use a lathe?” While there was a point, early in the project, that I may have had access to a six-foot lathe, the answer is, no, I did not use a lathe. Instead, I used only my table saw, hand planes, and a pile of sandpaper.

After making mortises for the legs, and routing the groove for the face panel (much easier on a square surface), I simply cut the corners off to produce an octagonal prism. I then cut those corners off to produce a 16-sided prism. Using a plane, I shaved the tips of those sixteen corners, then progressed through several grits of sandpaper. Because I decided completion was better than perfection, the rails are not perfect cylinders, but they’re close enough to please the eye and hand.

Homebrew serves multiple purposes
Homebrew serves multiple purposes
Gently adding a curve
Gently adding a curve

I would be remiss if I didn’t also mention the extra hardware that consumed portions of the living room for several weeks. The face panels in the headboard and footboard are not flat, but instead have a gentle curve to match the profile of the leg. To force the panel to maintain this curve, instead of fighting against it, I built the panel from two sheets of 1/4-inch plywood, glued together and squeezed in a bending form.

I did nearly all of the work in my simple basement shop, except for the very first cuts. Most of the wood I bought for this project was surfaced on at most one side. The crest rails are actually two halves of one very thick beam. The early work of surfacing and major cutting, I did in the MIT Hobby Shop. It’s a fantastic place that I never used as an undergrad, but I happily paid for a term of membership as an alumnus. Professional-grade jointer, planer, band saw, sander, etc. made the start of this project possible. It was also inspiring to see many undergrads taking advantage of what I had not, building everything from cabinetry to musical instruments.

I purchased the wood at The Woodery in Lunenberg, Mass. Despite having agreed to help a friend move, the operator of the yard stuck around an extra hour to help me sort through their stock to find exactly what I needed. With a great price to boot, I’ll likely be headed back there for my next project.

This design is not entirely my own. Beyond the influence of many woodworkers, both past and present, much of my final design is based on Jeff Miller‘s Sleigh Bed from his book Beds. If you’re planning to build a bed of any type, I recommend Jeff’s book, as it covers all of the basics (joinery, mattress support, etc.) with examples in many different styles.

What that next project may be, is up in the air. The next few months will likely be spent mostly on small projects that filled the queue while this bed filled the workshop. There are also holiday gifts to plan. After all of that, I’ll begin to consider the next large items on my list.

Roundtripping the HTTP Flowchart

Webmachine hackers are familiar with a certain flowchart representing the decisions made during the processing of an HTTP request. Webmachine was designed as a practical executable form of that flowchart.

It has long bugged many of the Webmachine hackers that this relationship is one-way, though. Webmachine was made from the graph, but the graph wasn’t made from Webmachine. I decided to change that in my evenings last week, while trying to take my mind off of Riak 1.0 testing.

This is a version of the HTTP flowchart that only a Webmachine hacker could love. It’s ugly and missing some information, but the important part is that it’s generated by parsing webmachine_decision_core.erl.

I’ve shared the code for generating this image in the gen-graph branch of my webmachine fork. Make sure you have Graphviz installed, then checkout that branch and run make graph && open docs/wdc_graph.png.

In addition to the PNG, you’ll also find a docs/wdc_graph.dot if you prefer to render to some other format.

If you’d really like to dig in, I suggest firing up an Erlang node and looking at the output of wdc_graph:parse("src/webmachine_decision_core.erl"):

[{v3b13, [ping],                     [v3b13b,503]},
 {v3b13b,[service_available],        [v3b12,503]},
 {v3b12, [known_methods],            [v3b11,501]},
 {v3b11, [uri_too_long],             [414,v3b10]},
 {v3b10, [allowed_methods,'RESPOND'],[v3b9,405]},
 {v3b9,  [malformed_request],        [400,v3b8]},
...

If you’ve looked through webmachine_decision_core at all, I think you’ll recognize what’s presented above: a list of tuples, each one representing the decision named by the first element, with the calls made to a resource module as the second element, and the possible outcomes as the third element. Call wdc_graph:dot/2 to convert those tuples to a DOT file.

There are a few holes in the generation. Some response codes are reached by decisions spread across the graph, causing long arrows to cross confusingly. The edges between decisions aren’t labeled with the criteria for following them. Some resource calls are left out (like those made from webmachine_decision_core:respond/1 and the response body producers and encoders). It’s good to have a nice list for future tinkering.

NerdKit Gaming: Part 2

If you were interested in my last bit of alternative code-geekery, you may also be interested to hear that I’ve pushed that NerdKit Gaming code farther. If you browse the github repository now, you’ll find that the game also includes a highscore board, saved in EEPROM so it persists across reboot. It also features a power-saving mode that kicks in if you don’t touch any buttons for about a minute. Key-repeat now also allows the player to hold a button down, instead of pressing it repeatedly, in order to move the cursor multiple spaces.

You may remember that I left of my last blog post noting that there wasn’t much left for the game until I could find a way to slim down the code to fit new things. So what allowed these new features to fit?

Well, I did find ways to slim down the code: I was right about making the game state global. But, I also re-learned a lesson that is at the core of hacking: check your base assumptions before fiddling with unknowns. In this case, my base assumption was the Makefile I imported from an earlier NerdKits project. While making the game state global saved a little better than 1k of space, changing the Makefile such that unused debugging utilities, such as uart, printf, scanf weren’t linked in saved about 6k.

In that learning, I also found that attempting to out-guess gcc’s “space” optimization is a losing game. Making the game state global had a positive effect on space, but making the button state global had a negative effect. Changing integer types would help in one place, but hurt in others. I’m not intimately familiar with the rules of that optimizer, so it felt like spining a wheel of chance choosing which thing to prod next.

You may notice that I ultimately returned the game state to a local variable, passed in and out of each function that needed it. The reason for this was testability. It’s simply easier to test something that doesn’t depend on global state. Once I had a bug that required running a few specific game states through these functions repeatedly, it just made sense to pay the price in program space in order to be able to write unit tests to cover some behaviors.

So now what’s next? This time, it’s not much until I buy a new battery. So much reloading and testing finally drained the original 9V. Once power is restored, I’ll probably dig into some new peripheral … maybe something USB?

Baseball + Riak Map/Reduce: the Movie

If you have been following my posts about using Riak’s map/reduce system to compute baseball statistics via the luwak_mr library, or if they’re still sitting in your ‘read later’ pile, you may be interested in a presentation I gave yesterday on the same topic.

Video of that presentation is available on Vimeo. It covers most of the content in the blog posts, while also providing a little extra background about why luwak_mr was necessary for the work.

NerdKit Gaming

Contrary to the evidence on this blog, not all of the code I write is in Erlang. It’s not even all web-based or dealing with distributed systems. In fact, this week I spent my evenings writing C for an embedded device.

I’ve mentioned NerdKits here before (affiliate link). This week I finally dug into the kit I ordered so long ago, and took it somewhere: gaming.

The result is a clone of a simple tile-swap matching game. I used very little interesting hardware outside the microcontroller and LCD — mostly just a pile of buttons. The purpose of this experiment was to test the capabilities of the little ATmega168 (and my abilities to program it).

I’ve put the code on github, if you’re interested in browsing. If you don’t have a NerdKit of your own to load it up on, I’ve also made a short demo video, and snapped a few up-close screenshots.

What did I learn? Mostly I remembered that writing a bunch of code to operate on a small amount of data can be just as fun as writing a bunch of code to operate on a large amount of data. Lots of interaction with the same few bytes from different angles has a different feel than the same operation repeated time and time again on lots of different data. I also learned that I’ve been spoiled by interactive consoles and fast compile/reload times. When it takes a minute or more to restart (after power cycles and connector un-re-plugging) and I don’t have an effectively infinite buffer to dump logs in, I think a little longer about each experiment.

So what’s next? Well, not much for this game, unless I slim down the code some more. Right now it compiles to 14310 bytes. Shortly before this, it was 38 bytes larger, and refused to load onto the microcontroller properly, since it plus the bootloader exceeds the 16K of flash memory available. My first attack would probably be to simply move the game board to a global variable instead of passing it as a function argument. The savings in stack-pushing should gain a little room.

If I were to make room for new operations, then a feature that saved a bit of state across power cycles would be a fun target. What’s a game without a high-score board?

Reading Code: Use Your Verbs

I’ve been reflecting on code quality lately. Partly that’s because I’ve been reading far more code than I’ve been writing. Partly it’s because the most recent code I was writing was intended primarily for reading, and only incidentally for execution, in the most literal way: it was instructional, not application-supporting.

And so it is that I’ve recently reaffirmed my conviction that code’s quality is primarily a function of its readability. Readability is of primary importance because code must be able to be understood in order to be used, and the way to understand it best is to read it.

However, I think I can be more specific about one component of readability that holds sway over the rest: naming. Partially the quality of each name, but also the ratio of named to unnamed things. But most important of all, the ratio of named to unnamed verbs.

I first realized this several years ago, while hacking in the middle of a complex, distributed, Java-based system. At one point, I had spent days diving through spaghetti, and finally found the core of the system … and it was beautiful. Not just the best Java code I’d ever seen, but possibly the cleanest code, period. Comparing it to the ugly code I had dug through, I found that its cleanliness derived from the fact that each interesting operation (or “verb”) was segregated into its own named function. Some of those functions called others of those functions, but it was always just one operation described in each.

Later, coincidentally on the same project, I had reason to spend several weeks not in Java, a language I knew very well, but in Perl, Python, and Bash, languages with which I was less familiar. I wrote and modified code very carefully in those languages, making sure that I could test each step as I went along. As that bit of hacking finished, I returned to Java, and found that my style had changed. I was now writing Java in a very careful, easily-testable manner. When I stepped back, I realized that the easily-understood form of my new Java code shared something with that beautiful core I had found earlier: each function described exactly one operation.

I’ll demonstrate what I mean with a concrete example. The code below is very similar to code I was hacking recently. The labels have been changed to protect the innocent, even though I think the innocent is me.

The set_properties function expects a token and a collection of properties (keys with matching values) to store for the token. New properties should overwrite old properties of matching keys, but old values for keys that are not specified should remain unchanged. For example, if the token “foo” had the properties [{a,1},{b,1}], and I called set_properties with the new properties [{a,2},{c,2}], then after set_properties finishes, the token “foo” should have the properties [{a,2},{b,1},{c,2}] (the new values for a and c plus the old value for b).

set_properties(Token, NewProperties) ->
   OldProperties = get_properties(Token),
   NewKeys = [ K || {K, _} <- NewProperties ],
   FilteredProperties = [ P || P={K, _} <- OldProperties,
                               not lists:member(K, NewKeys) ],
   set_properties_internal(Token, FilteredProperties ++ NewProperties).
    
Fig. 1: The Beginning

The code in Figure 1 is where I started. This code is correct: it conforms to the spec given, passes all tests (indeed, has been in production, working, for over a year). But, it is also bad code. The hint why is the NewKeys variable. It has little to do with setting new properties; it’s merely an artifact of cleaning up old properties. It’s an indication that the two list comprehensions that reference it are really an unnamed verb separate from set_properties.

set_properties(Token, NewProperties) ->
   OldProperties = get_properties(Token),
   MergedProperties = merge_properties(NewProperties, OldProperties).
   set_properties_internal(Token, MergedProperties).

merge_properties(Keep, Toss) ->
   KeepKeys = [ K || {K, _} <- Keep ],
   FilteredToss = [ P || P={K, _} <- Toss,
                         not lists:member(K, KeepKeys) ],
   FilteredToss ++ Keep.
    
Fig. 2: Naming the Verb

I propose that the code in Figure 2 is an improvement upon the code in Figure 1. The set_properties function now says just exactly what it’s going to do: lookup the old properties, merge them with the new properties, and store the result. The details about how the merge is performed, the unnamed verb in Figure 1, have been relocated to a new function, named merge_properties. The intermediate list of keys is still produced, but it’s now obvious that it’s just part of the merging process.

set_properties(Token, NewProperties) ->
   OldProperties = get_properties(Token),
   MergedProperties = merge_properties(NewProperties, OldProperties),
   set_properties_internal(Token, MergedProperties).

merge_properties(Keep, Toss) ->
   lists:ukeymerge(1, lists:ukeysort(1, Keep), lists:ukeysort(1, Toss)).
    
Fig. 3: Using an Existing Name

Figure 3 is a demonstration of part of the reason that MIT changed the 6.001 curriculum. There was no need to write those list comprehensions. Someone had already written the equivalent and named it. It is far clearer to use that named operation than to reimplement. The confusion about why NewKeys was created has been removed, and so has the need to decrypt the other list comprehension.

set_properties(Token, NewProperties) ->
   OldProperties = get_properties(Token),
   MergedProperties = lists:ukeymerge(1,
                         lists:ukeysort(1, NewProperties),
                         lists:ukeysort(1, OldProperties)),
   set_properties_internal(Token, MergedProperties).   
    
Fig. 4: Breaking Context

It’s a valid question to ask why I didn’t recommend jumping straight from Figure 1 to Figure 4, instead of ending up at Figure 3. It’s true that Figure 4 is a large improvement on Figure 1, but the answer is that even though lists:ukeymerge/3 is a named verb, it’s a verb with less context than merge_properties in my module. The context is richer than this snippet suggests, because there is at least one other function in this module that needs to perform the same operation. Also, to reference 6.001 again, “Abstraction barrier!” Why does set_properties need to know the data structure I’m using?

set_properties(Token, NewProperties) ->
   set_properties_internal(
      Token, merge_properties(NewProperties, get_properties(Token))).

merge_properties(Keep, Toss) ->
   lists:ukeymerge(1, lists:ukeysort(1, Keep), lists:ukeysort(1, Toss)).
    
Fig. 5: Anonymous Nouns

Another valid question is why I didn’t continue on to Figure 5 after Figure 3. In truth, I did consider it. My eye sees less clutter, but having discussed this exact choice with many coworkers, I’ve learned that others don’t. It also goes against the grain of what this post has been advocating: while I’ve worked to name my verbs, Figure 5 anonymized my nouns. There’s a practical reason to keep names for nouns around: printf debugging. Unless I have a very nice macro handy that I can wrap around one of the function calls in-place in Figure 5, I’m forced to copy one of those calls to some other place, and possibly even give it a name, before I can wrap my print statement around it. In Figure 3, the names are already there; all I have to do is use them.

What else could be improved in Figure 3? Plenty: “merge” is a bit generic and over-used; “properties” is long, noisy, and redundant in-context. Is my omission of names for the sorted lists in merge_properties/2 hypocritical? Probably. Readability is a subjective, human measure. In multiple projects and languages, I’ve identified verb-naming as important in my judgement of a code’s readability. Maybe writing that fact down will help me remember to think about it in new code I write.

Baseball + Riak Map/Reduce, Round 2

If you enjoyed my last post on the Basho Blog, about computing baseball stats using Riak’s map/reduce, you may also enjoy my followup post about dealing with (or avoiding) records that have been split across block boundaries.

Baseball + Riak Map/Reduce

The luwak_mr tool I wrote about last weekend kept my imagination chugging this week. The result? I’ve learned a bit about baseball, and written code to compute batting average using luwak_mr (and Riak map/reduce, of course).

Learn how it works on the Basho Blog.

Follow

Get every new post delivered to your Inbox.