I'm not super into the Internet-of-Things. There are no wifi lightbulbs, electronic locks, or smart thermostats in my house. But, I'm a homebrewer, and that means I love new ways to get data about my beer. I backed The BeerBug on Kickstarter, and I've used it on a number of batches since early 2014.
The data my BeerBug provides is simple, but interesting: air temperature and specific gravity, measured once per minute. It gives me a pretty good idea of when a beer has finished or stalled.
The user experience leaves something to be desired, though. The website is clunky, and was down for a month or more recently. The mobile app is just a web view. There is no way to use the device without the website.
So, I have two goals over the next few months. The first is to extract all of the data I have recorded with my BeerBug, and the second is to find an alternative. This post covers the first goal, and the next will begin to explore the second.
The BeerBug offers an API ... that only covers active brewing, not history. Beer pages allegedly offer CSV and XML data download, but the links haven't worked in months. You can view graphs of historical brews on the website, though, so they have the ability to fetch that data.
Pulling up the Chrome web inspector and visiting a beer page, there is an XHR for a "graph.php" that returns JSON to draw the graph. Try as I might, I haven't been able to construct a curl command to get the same data - it always came through with "0" or "null" in several fields. There's almost certainly some header I'm missing, but I've taken an alternate route.
The network tab of Chrome's web inspector will let you "Save as HAR with Content." This exports a JSON file will all the information the inspector is showing. Lucky for me, this includes the content of the graph.php XHR response. So, switching the graph view from "25 points" to "all" and waiting for the new graph.php request to complete, then saving as HAR has captured my data.
The data from the XHR is the last in the log entries, so it's easy to extract with jq:
$ jq ".log.entries[-1].response.content.text | fromjson" \<br />
export-oatmeal-stout-jan-2016.har > export-oatmeal-stout-jan-2016.json<br />
Now I can start to explore the data:
$ jq ". | keys" export-oatmeal-stout-jan-2016.json
[
"al",
"batt",
"dates",
"degrees",
"ext",
"plato",
"platod",
"sg",
"success",
"temp",
"temp2"
]
Almost all of these fields are arrays with one entry per measurement:
- al: alcohol percentage
- batt: battery voltage (volts)
- dates: date of measurement (comma-separated strings year,month,day,hour,minute,second - not width-padded, zero-based month index, local timezone)
- platod: degrees plato
- sg: specific gravity
- temp: air temperature (either Fahrenheit or Celcius, depending on value of "degrees" field)
- temp2: probe temperature
Non-array fields:
- degrees: what units "temp" and "temp2" are in ("F" for Fahrenheit, and I assume "C" for Celcius, but I haven't checked)
- ext: unknown
- plato: unknown
- success: unknown
Just a bit of data checking: I started the beer on January 23, 2016, and finished it on February 8:
$ jq ".dates[0], .dates[-1]" export-oatmeal-stout-jan-2016.json
"2016,0,23,18,35,3"
"2016,1,08,15,18,3"
Its specific gravity started about where I normally start my beers, and ended a little below where I normally finish them:
$ jq ".sg[0], .sg[-1]" export-oatmeal-stout-jan-2016.json
1.0568
1.0082
That means it may have a 6.4% alcohol content by volume:
$ jq ".al[0], .al[-1]" export-oatmeal-stout-jan-2016.json
0
6.4
And finally, it was kept in nice cool range (`add / length` is jq for "average"):
$ jq ".temp | max, min, add / length" export-oatmeal-stout-jan-2016.json
71.18
63.4
65.68423989795319
Neat. Let's compare all the beers I exported:
# extract all xhr data
$ for x in export*.har; \
do jq ".log.entries[-1].response.content.text | fromjson" $x \
> ${x/har/json}; \
done
# extract basic data
$ for x in export*.json; \
do echo $x && jq -c '{"sg":.sg[0],"fg":.sg[-1],"abv":.al[-1],"temp":{"min":.temp|min,"max":.temp|max,"avg":(.temp|add/length)}}' $x; \
done
export-abbey-oct-2015.json
{"sg":1.0498,"fg":1.4284,"abv":0,"temp":{"min":69.74,"max":79.96,"avg":72.70824454043661}}
export-beechwood-smoke-may-2014.json
{"sg":1.0511,"fg":0.9935,"abv":7.5,"temp":{"min":71.8,"max":83,"avg":75.40845794392524}}
export-butternut-stout-nov-2014.json
{"sg":1.0529,"fg":1.3635,"abv":0,"temp":{"min":65.36,"max":74.41,"avg":69.15657534246593}}
export-ipa-may-2015.json
{"sg":1.0475,"fg":0.9946,"abv":6.7,"temp":{"min":68.81,"max":80.21,"avg":71.19772108108131}}
export-mead.json
{"sg":1.115,"fg":1.0389,"abv":10,"temp":{"min":61,"max":70.84,"avg":65.09618010573946}}
export-oatmeal-stout-jan-2016.json
{"sg":1.0568,"fg":1.0082,"abv":6.4,"temp":{"min":63.4,"max":71.18,"avg":65.68423989795319}}
export-oatmeal-stout-nov-2015.json
{"sg":1.0639,"fg":1.0108,"abv":7,"temp":{"min":63.66,"max":77.25,"avg":69.64541020966313}}
export-oatmeal-stout-sep-2014.json
{"sg":1.0499,"fg":0.9973,"abv":7.3,"temp":{"min":72.3,"max":81.8,"avg":76.59252173913043}}
export-pumpkin-ale-nov-2015.json
{"sg":1.0529,"fg":1.0134,"abv":5.2,"temp":{"min":63.37,"max":70.69,"avg":66.15414939483689}}
There is quite a bit more analysis that should be done on this data. For example, I know that the specific gravity jumps around quite a lot. It is measured by a hall-effect sensor capturing the weight of a plumb in the beer, and so it's a bit touchy about temperature changes and carbonation bubbles from active yeast. Those simple stats about the temperature (min, max, mean) do not really tell the whole story.
But, I'm fairly well convinced that I now have a copy of my recorded data. What is the path forward? Find out in part two.
Categories: Development
Post Copyright © 2017 Bryan Fink