Beer IoT (Part 1)

I’m not super into the Internet-of-Things. There are no wifi lightbulbs, electronic locks, or smart thermostats in my house. But, I’m a homebrewer, and that means I love new ways to get data about my beer. I backed The BeerBug on Kickstarter, and I’ve used it on a number of batches since early 2014.

The data my BeerBug provides is simple, but interesting: air temperature and specific gravity, measured once per minute. It gives me a pretty good idea of when a beer has finished or stalled.

The user experience leaves something to be desired, though. The website is clunky, and was down for a month or more recently. The mobile app is just a web view. There is no way to use the device without the website.

So, I have two goals over the next few months. The first is to extract all of the data I have recorded with my BeerBug, and the second is to find an alternative. This post covers the first goal, and the next will begin to explore the second.

The BeerBug offers an API … that only covers active brewing, not history. Beer pages allegedly offer CSV and XML data download, but the links haven’t worked in months. You can view graphs of historical brews on the website, though, so they have the ability to fetch that data.

Pulling up the Chrome web inspector and visiting a beer page, there is an XHR for a “graph.php” that returns JSON to draw the graph. Try as I might, I haven’t been able to construct a curl command to get the same data – it always came through with “0” or “null” in several fields. There’s almost certainly some header I’m missing, but I’ve taken an alternate route.

The network tab of Chrome’s web inspector will let you “Save as HAR with Content.” This exports a JSON file will all the information the inspector is showing. Lucky for me, this includes the content of the graph.php XHR response. So, switching the graph view from “25 points” to “all” and waiting for the new graph.php request to complete, then saving as HAR has captured my data.

The data from the XHR is the last in the log entries, so it’s easy to extract with jq:

$ jq ".log.entries[-1].response.content.text | fromjson" \
  export-oatmeal-stout-jan-2016.har > export-oatmeal-stout-jan-2016.json

Now I can start to explore the data:

$ jq ". | keys" export-oatmeal-stout-jan-2016.json
[
 "al",
 "batt",
 "dates",
 "degrees",
 "ext",
 "plato",
 "platod",
 "sg",
 "success",
 "temp",
 "temp2"
]

Almost all of these fields are arrays with one entry per measurement:

  • al: alcohol percentage
  • batt: battery voltage (volts)
  • dates: date of measurement (comma-separated strings year,month,day,hour,minute,second – not width-padded, zero-based month index, local timezone)
  • platod: degrees plato
  • sg: specific gravity
  • temp: air temperature (either Fahrenheit or Celcius, depending on value of “degrees” field)
  • temp2: probe temperature

Non-array fields:

  • degrees: what units “temp” and “temp2” are in (“F” for Fahrenheit, and I assume “C” for Celcius, but I haven’t checked)
  • ext: unknown
  • plato: unknown
  • success: unknown

Just a bit of data checking: I started the beer on January 23, 2016, and finished it on February 8:

$ jq ".dates[0], .dates[-1]" export-oatmeal-stout-jan-2016.json
"2016,0,23,18,35,3"
"2016,1,08,15,18,3"

Its specific gravity started about where I normally start my beers, and ended a little below where I normally finish them:

$ jq ".sg[0], .sg[-1]" export-oatmeal-stout-jan-2016.json
1.0568
1.0082

That means it may have a 6.4% alcohol content by volume:

$ jq ".al[0], .al[-1]" export-oatmeal-stout-jan-2016.json
0
6.4

And finally, it was kept in nice cool range (`add / length` is jq for “average”):

$ jq ".temp | max, min, add / length" export-oatmeal-stout-jan-2016.json
71.18
63.4
65.68423989795319

Neat. Let’s compare all the beers I exported:

# extract all xhr data
$ for x in export*.har; \
    do jq ".log.entries[-1].response.content.text | fromjson" $x \
    > ${x/har/json}; \
  done
# extract basic data
$ for x in export*.json; \
    do echo $x && jq -c '{"sg":.sg[0],"fg":.sg[-1],"abv":.al[-1],"temp":{"min":.temp|min,"max":.temp|max,"avg":(.temp|add/length)}}' $x; \
  done
export-abbey-oct-2015.json
{"sg":1.0498,"fg":1.4284,"abv":0,"temp":{"min":69.74,"max":79.96,"avg":72.70824454043661}}
export-beechwood-smoke-may-2014.json
{"sg":1.0511,"fg":0.9935,"abv":7.5,"temp":{"min":71.8,"max":83,"avg":75.40845794392524}}
export-butternut-stout-nov-2014.json
{"sg":1.0529,"fg":1.3635,"abv":0,"temp":{"min":65.36,"max":74.41,"avg":69.15657534246593}}
export-ipa-may-2015.json
{"sg":1.0475,"fg":0.9946,"abv":6.7,"temp":{"min":68.81,"max":80.21,"avg":71.19772108108131}}
export-mead.json
{"sg":1.115,"fg":1.0389,"abv":10,"temp":{"min":61,"max":70.84,"avg":65.09618010573946}}
export-oatmeal-stout-jan-2016.json
{"sg":1.0568,"fg":1.0082,"abv":6.4,"temp":{"min":63.4,"max":71.18,"avg":65.68423989795319}}
export-oatmeal-stout-nov-2015.json
{"sg":1.0639,"fg":1.0108,"abv":7,"temp":{"min":63.66,"max":77.25,"avg":69.64541020966313}}
export-oatmeal-stout-sep-2014.json
{"sg":1.0499,"fg":0.9973,"abv":7.3,"temp":{"min":72.3,"max":81.8,"avg":76.59252173913043}}
export-pumpkin-ale-nov-2015.json
{"sg":1.0529,"fg":1.0134,"abv":5.2,"temp":{"min":63.37,"max":70.69,"avg":66.15414939483689}}

There is quite a bit more analysis that should be done on this data. For example, I know that the specific gravity jumps around quite a lot. It is measured by a hall-effect sensor capturing the weight of a plumb in the beer, and so it’s a bit touchy about temperature changes and carbonation bubbles from active yeast. Those simple stats about the temperature (min, max, mean) do not really tell the whole story.

But, I’m fairly well convinced that I now have a copy of my recorded data. What is the path forward? Find out in part two.

The New Shop Works

You should see the look you get when you tell a moving crew that you’re taking three 200lbs slabs of slate with you on your move across the country. Maybe it was just the fact that we were standing in a fieldstone basement, and they were suddenly wondering how many of the other rocks in few were coming along. At least there was no further argument when I added, “And this pile of wood too.”

Some time in 2005 I was tipped off that an old pool table was being thrown out. It had lived in a cabin, without climate control. The felt was shot, and not much better could be said of the wood structure beneath. I found myself drawn to the idea of the slate between the two, though. Probably there was a subconscious dream of refinishing a pool table for my own house, but there were definitely also thoughts of chalkboards and such.

And so, the chunks of table top made it down one snowy January slope, and up another, into the back of my truck, back to my home, and into my basement … where they sat for seven years. When I first felt the mass of them, the chalkboard dreams vanished — who would feel safe attaching that to a wall? It took a bit of time to come up with other plans.

The first test came right after building a bed. Much smaller pieces were required, which both alleviated the weight concerns, and also meant there was plenty of material to experiment (i.e. fail and retry) with. I had no stonework experience, but after reading, watching youtube videos, and playing around, I cleaved two pieces that made pretty, textured tops for our nightstands.

Then the slabs sat dormant again. Shortly after my previous blog post, we left Greater Boston and moved to the San Francisco Bay Area. Thus, the sidelong looks from the movers.

While we moved most of the house, one thing we didn’t move was the coffee table. That was a varnished pine concoction that I built in college, and we found it a nice home instead of bringing it along. That opened a hole in the living room, and what else could I possibly see but the weighty, smooth, shady, cool slice of slate?

Some sketches, some planning, a load of lumber, and a few months of weekends later…

IMG_1169

I’m super happy with how this turned out. The top is cut down to 21 by 42 inches, with some natural cleaving around the edge for detail. It’s finished with Glaze’n’Seal, to keep stains off. The base is mahogany, hand-planed planks glued edge-to-edge, so the weight is supported in the direction of the grain. The wood is finished with my trusty mix of mineral oil and beeswax.

This was the first project I’ve done where I didn’t have all of the dimensions nailed down before starting. Given the piece of slate, and the basic height and shape of the base, I calculated the maximum amount of wood I could need, and started there.

I found the wood at Global Wood Source. They had stacks of beautiful species, and they were very friendly and helpful. I settled on mahogany mostly for its color, which I thought would contrast with the grey slate without being jarring.

I probably should have bought a planer at this point. But instead I hauled out my hand plane and water stones. Three sessions, each a few hours in length, and I had some roughly jointed planks.

I cut these to length – 16 inches – and then determined how many I would need to span from one corner to the other of my slab. After a bit more edge cleanup, the gluing began.

The cross-pieces and shelves came together in a similar manner, and once they each had their basic shape, the work began to fit them together.

Sanding was the wonderful process it always is – tiring, dirty, but revealing. With each new grit, more of the figure of the grain became visible. And some day I’ll spend more time learning how to capture it on camera properly…

I risked doing all five pieces at once in the final glue up. It seemed like the best bet for making sure the whole unit was flat and stable. Luckily it seems to have worked.

If each successive stage of sanding makes the grain more beautiful, then the first coat of polish is the ultimate sanding. Though the dust was undoubtedly red, the pieces themselves had been quite pale to this point. When the oil and beeswax hit them, though, they popped.

Meanwhile, there was stonework. I found a simple wet tile saw to work fairly well, even if it did produce large clumps of clay. I took the slab to size early in the project, to be able to double-check and gauge true dimensions.

Once the base was done, I cut grooves in the bottom of the slab to keep it from moving around. This was by far the dirtiest part – the saw flung tiny bits of slate all over. I was glad to be wearing a dust mask and safety goggles.

The final detail on the top was a cleaved edge. A few minutes of scoring a line along the edge, 1/4 inch from the top, and then just a fun time tapping a cold chisel.

A bit of sanding on the top face was required to remove the dusty, scratched surface, and to remove sharp edges. Coats of Glaze’n’Seal went on with drama, and now the piece sits in our living room. It passed the dinner test. 🙂

IMG_1174

My Favorite Moment of 2013

It’s the last day of 2013, and I’m supposed to be finishing preparations for a cross-country move. But instead, I really want to recount my favorite moment of this past year.

On Friday, October 11, 2013, MIT’s Hobby Shop held a celebration to commemorate its 75th anniversary. The hobby shop is a place for the MIT community (students, faculty, alumni, and such) to … well, practice *manus* after stretching their *mens*. It’s a large room, filled with benches, power tools, and hand tools for working wood, metal, plastic, etc.

People use the Hobby Shop to build … things. Equipment for lab projects, musical instruments, furniture, signs, or whatever else they might dream. I was (sadly) not a member in college, but joined later to learn and use their large machinery when starting my bed.

The celebration in October included many member projects on display, one of which was a camera. Biyeun, its builder and user, gave a presentation about making and using her creation. In her introduction, she explained her discovery of view cameras and her instantaneous reaction: “I must build that.”

As I nodded my head in understanding of her sentiment, I saw heads all around the room do likewise. Building a machine gives you a different understanding of it that no variety of use ever will. Just a taste of such knowledge can cause everyday objects to practically scream at you forever afterward, “Imagine what it’s like to create me.” I knew that everyone nodding had heard that call.

The dean of student life, Chris Colombo, spoke as well. He was not a member of the Hobby Shop, but had good friends there. He expressed awe for the projects like Biyuen’s camera, that he had seen leave the shop, and a few minutes into his speech said something like, “I wish I knew how to build something like that.” As he took a breath afterward, I could just feel every shop member in the room struggle to restrain themselves from walking onto the stage, grabbing Chris by the elbow, and dragging him to the shop, to teach him how. “C’mon, I’ll show you,” were the words on every lip.

Realizing that I was surrounded by people that not only had wanted to know, and then spent time doing and learning, but now also wanted to show and teach, was my favorite moment in 2013. Finding people that are curious is not terribly hard. Finding those that will follow through on their curiosity can sometimes seem rare. But, finding one who actually wants to share what he or she has learned, by answering the endless naive questions of a beginner, is like winning the lottery. To be standing in a room full of such individuals was overwhelming.

Hobbies -= 1

I shut down a hobby today. BeerRiot, the site I started over six years ago, is now closed. I’m keeping the domain active, because I’ve used the name in other places, but browsers will see only a static archive of what used to be there.

BeerRiot began as an experiment. I wanted to learn about Erlang, and I needed a project to drive my curiosity. It worked, and I learned a good deal about modern web application development in the process. In fact, I learned enough about both that, through blogging about my progress, I was able to join up with a smart team and work in Erlang on web apps professionally.

In fact, even after the experiment paid off, BeerRiot remained my sandbox. New webservers, new storage techniques, new rendering processes, new API designs … I was able to practice with them all in a live setting before attempting to pull an entire team of engineers toward any of them.

So why would I give up my playground? Simply put: I don’t play there any more. My interests have moved on, and it’s time to remove the mental clutter of the service existing (no matter it’s reliability). Were the virtual server some physical object, I’d be putting it on a garage sale. As it is not, I will instead throw a tarball on a backup disk, and laugh when I find it in a few years.

What’s next? On the code side, more focus on that smart team and profession Erlang work I mentioned. On the hobby side … definitely not another web app. I’ll keep this blog up. No promises on changes to its post frequency, but readers will be among the first to know when I find a new thing.

Cheers.

I built a maple sleigh bed.

Some of you know that beyond beer and coding, I’m also an active woodworker. In this post, I’m excited to share the completion of my most recent project. Approximately 18 months after buying the first lumber, I’m now sleeping in in my hand-made hard maple sleigh bed.

Some of you know that beyond beer and coding, I’m also an active woodworker. In this post, I’m excited to share the completion of my most recent project. Approximately 18 months after buying the first lumber, I’m now sleeping in in my hand-made hard maple sleigh bed.

The finished bed
The finished bed

It’s not my first piece of furniture, but it is, by far, the largest and most intricate to date. Despite some amateur imperfections (or are we calling those “artisanal qualities” these days?), I’m quite happy with how it turned out. It looks good, it sits straight, the matress fits, and it doesn’t squeak!

Slats hold the mattress
Slats hold the mattress

The matress is supported by fifteen poplar slats, each 4 inches wide by 3/4 inches thick, with an inch of space between them. That is to say, there’s no box spring. We bought the matress about a year ago, and we have used a futon frame (also a slatted frame), to support it since then. At “full” size, these slats seem to be fairly firm, but not hard. We like it.

Raw below, one coat of finish above
Raw below, one coat of finish above

The finish is a mixture of one part mineral oil to 4-5 parts beeswax, by volume. It’s not a hard, take-a-beating kind of finish, and it will need to be reapplied from time to time, but we couldn’t resist the beautiful natural color of the maple, the sweet honey smell, and the smooth matte texture. We don’t expect it to need to withstand much more than our touch and the seasonal humidity change anyway. Rumor also has it that the finish and wood should change color with exposure to the sun as the years go by, which will add a great living element to a long-loved piece of furniture.

Tenons on the footboard
Tenons on the footboard

You may notice that there is no metal hardware visible. The headboard and footboard are mortise-and-tenon boxes around a floating plywood panel. Glue and a good fit is all that’s holding them together.

Hidden bolt joinery
Hidden bolt joinery

Fear not, though, for I am not crazy enough to glue up a piece of furniture that cannot later be removed from a room. The side rails are attached to the headboard and footboard via bolts, but in a sneaky way. The bolts are recessed into the side rail from the inside. They protrude from the end of the side rail, and then pierce the headboard and footboard through a hole on each inner face. A square nut is captured in the tenon of the lower cross rail, to secure the end of the bolt. Wood pins also protrude from the end of the side rail, and slot into holes in the headboard and footboard, to prevent the side rail from spinning around the bolt.

Cleats, glued and screwed
Cleats, glued and screwed

Beyond connecting the headboard and footboard to each other, the side rail also holds the cleats, which support the slats for the matress. The cleats are attached with good, old-fashions glue-and-screws construction, to ensure they never pry themselves off.

Square Octagonal Sixteen sides Nearly round Polished
Progressively rounder

The crest rails atop the headboard and footboard are defining features of the bed. The first question I’m always asked about them is, “Did you use a lathe?” While there was a point, early in the project, that I may have had access to a six-foot lathe, the answer is, no, I did not use a lathe. Instead, I used only my table saw, hand planes, and a pile of sandpaper.

After making mortises for the legs, and routing the groove for the face panel (much easier on a square surface), I simply cut the corners off to produce an octagonal prism. I then cut those corners off to produce a 16-sided prism. Using a plane, I shaved the tips of those sixteen corners, then progressed through several grits of sandpaper. Because I decided completion was better than perfection, the rails are not perfect cylinders, but they’re close enough to please the eye and hand.

Homebrew serves multiple purposes
Homebrew serves multiple purposes
Gently adding a curve
Gently adding a curve

I would be remiss if I didn’t also mention the extra hardware that consumed portions of the living room for several weeks. The face panels in the headboard and footboard are not flat, but instead have a gentle curve to match the profile of the leg. To force the panel to maintain this curve, instead of fighting against it, I built the panel from two sheets of 1/4-inch plywood, glued together and squeezed in a bending form.

I did nearly all of the work in my simple basement shop, except for the very first cuts. Most of the wood I bought for this project was surfaced on at most one side. The crest rails are actually two halves of one very thick beam. The early work of surfacing and major cutting, I did in the MIT Hobby Shop. It’s a fantastic place that I never used as an undergrad, but I happily paid for a term of membership as an alumnus. Professional-grade jointer, planer, band saw, sander, etc. made the start of this project possible. It was also inspiring to see many undergrads taking advantage of what I had not, building everything from cabinetry to musical instruments.

I purchased the wood at The Woodery in Lunenberg, Mass. Despite having agreed to help a friend move, the operator of the yard stuck around an extra hour to help me sort through their stock to find exactly what I needed. With a great price to boot, I’ll likely be headed back there for my next project.

This design is not entirely my own. Beyond the influence of many woodworkers, both past and present, much of my final design is based on Jeff Miller‘s Sleigh Bed from his book Beds. If you’re planning to build a bed of any type, I recommend Jeff’s book, as it covers all of the basics (joinery, mattress support, etc.) with examples in many different styles.

What that next project may be, is up in the air. The next few months will likely be spent mostly on small projects that filled the queue while this bed filled the workshop. There are also holiday gifts to plan. After all of that, I’ll begin to consider the next large items on my list.

Roundtripping the HTTP Flowchart

It has long bugged many of the Webmachine hackers that this relationship with Alan Dean’s HTTP flowchart is one-way. Webmachine was made from that graph, but that graph wasn’t made from Webmachine. I decided to change that in my evenings last week.

Webmachine hackers are familiar with a certain flowchart representing the decisions made during the processing of an HTTP request. Webmachine was designed as a practical executable form of that flowchart.

It has long bugged many of the Webmachine hackers that this relationship is one-way, though. Webmachine was made from the graph, but the graph wasn’t made from Webmachine. I decided to change that in my evenings last week, while trying to take my mind off of Riak 1.0 testing.

This is a version of the HTTP flowchart that only a Webmachine hacker could love. It’s ugly and missing some information, but the important part is that it’s generated by parsing webmachine_decision_core.erl.

I’ve shared the code for generating this image in the gen-graph branch of my webmachine fork. Make sure you have Graphviz installed, then checkout that branch and run make graph && open docs/wdc_graph.png.

In addition to the PNG, you’ll also find a docs/wdc_graph.dot if you prefer to render to some other format.

If you’d really like to dig in, I suggest firing up an Erlang node and looking at the output of wdc_graph:parse("src/webmachine_decision_core.erl"):

[{v3b13, [ping],                     [v3b13b,503]},
 {v3b13b,[service_available],        [v3b12,503]},
 {v3b12, [known_methods],            [v3b11,501]},
 {v3b11, [uri_too_long],             [414,v3b10]},
 {v3b10, [allowed_methods,'RESPOND'],[v3b9,405]},
 {v3b9,  [malformed_request],        [400,v3b8]},
...

If you’ve looked through webmachine_decision_core at all, I think you’ll recognize what’s presented above: a list of tuples, each one representing the decision named by the first element, with the calls made to a resource module as the second element, and the possible outcomes as the third element. Call wdc_graph:dot/2 to convert those tuples to a DOT file.

There are a few holes in the generation. Some response codes are reached by decisions spread across the graph, causing long arrows to cross confusingly. The edges between decisions aren’t labeled with the criteria for following them. Some resource calls are left out (like those made from webmachine_decision_core:respond/1 and the response body producers and encoders). It’s good to have a nice list for future tinkering.

NerdKit Gaming: Part 2

If you were interested in my last bit of alternative code-geekery, you may also be interested to hear that I’ve pushed that NerdKit Gaming code farther. If you browse the github repository now, you’ll find that the game also includes a highscore board, saved in EEPROM so it persists across reboot. It also features a power-saving mode that kicks in if you don’t touch any buttons for about a minute. Key-repeat now also allows the player to hold a button down, instead of pressing it repeatedly, in order to move the cursor multiple spaces.

You may remember that I left of my last blog post noting that there wasn’t much left for the game until I could find a way to slim down the code to fit new things. So what allowed these new features to fit?

Well, I did find ways to slim down the code: I was right about making the game state global. But, I also re-learned a lesson that is at the core of hacking: check your base assumptions before fiddling with unknowns. In this case, my base assumption was the Makefile I imported from an earlier NerdKits project. While making the game state global saved a little better than 1k of space, changing the Makefile such that unused debugging utilities, such as uart, printf, scanf weren’t linked in saved about 6k.

In that learning, I also found that attempting to out-guess gcc’s “space” optimization is a losing game. Making the game state global had a positive effect on space, but making the button state global had a negative effect. Changing integer types would help in one place, but hurt in others. I’m not intimately familiar with the rules of that optimizer, so it felt like spining a wheel of chance choosing which thing to prod next.

You may notice that I ultimately returned the game state to a local variable, passed in and out of each function that needed it. The reason for this was testability. It’s simply easier to test something that doesn’t depend on global state. Once I had a bug that required running a few specific game states through these functions repeatedly, it just made sense to pay the price in program space in order to be able to write unit tests to cover some behaviors.

So now what’s next? This time, it’s not much until I buy a new battery. So much reloading and testing finally drained the original 9V. Once power is restored, I’ll probably dig into some new peripheral … maybe something USB?