Archive for the ‘Development’ Category


A year in the making, almost completely rewritten, I can’t bear to hold it back any longer: today I release the new BeerRiot. Here’s a synopsis of the changes for you:

Old Tech New Tech
Erlyweb (Yaws) Webmachine (Mochiweb)
Erlydb + MySQL Hand-coded models + Mnesia + Apache Solr
ErlTL jQuery

I’ll probably write a blog post about each of those rows sometime in the near future. It should be said though, the my motivation in this rewrite was not to abandon Erlyweb. Rather each piece was a deliberate attempt to get practice on something we were using at work. Erlyweb’s great, but Webmachine has a different feel. MySQL can store data fine, but it’s quite different from the key-value store I hack against all day. ErlTL’s pretty nice as templating languages go, but I needed more DOM experience.

Luckily, the new technologies are also very nice. Webmachine forces you to become more familiar with the ins and outs of HTTP, but after writing a few resources, it becomes natural and quick to create new ones. Storing data in Erlang format in Mnesia is heavenly, and Solr has drastically improved search functionality. jQuery makes JavaScript in the browser far less painful.

In doing this rewrite, it’s been a real eye-opener to dig back through my early Erlang code. It wasn’t terrible, but having worked with other serious Erlang hackers all year, I notice the difference in the code I write now. The site should be much more stable now – Local/maps may even stay up for more than an hour. 😉

In that vein, though, I request that you not judge the JS running the site too harshly just yet. Just like my early Erlang was ugly, I can now tell that my early JS was ugly as well. That will be getting some cleanup soon, but I just couldn’t stand delaying the release for it.

So, go poke it and let me know what you think!

WWW is back

Unfortunately, I only found out this morning that some of you probably thought I had given up on BeerRiot. A month or so ago, I started hosting another domain on the same server, and thought I’d set up Yaws to handle that properly. Unfortunately, I botched it bad enough that while still worked, didn’t. Sorry!

Anyway, things are fixed now, so if you like prepending “www.”, welcome back!

You may have also questioned the lack of new feature announcements of late. The reason behind this is a giant rewrite I’ve been working on all year. More details in a few weeks, but it’s almost an entire redesign that touches every level. At the same time, work has been crazy busy all year. I’m excited to share the new stuff with you all, though, so keep me in your RSS feed, so you don’t miss the announcement.

Denormalization, Processes

If you read the news, you’ll know that tuneups are happening behind the scenes of BeerRiot. If you came to this blog after reading that story, you’re wondering what, exactly, they are.

If I’m not feeling particularly communication-challenged, I’ll be able to explain them to you. 😉

The first tuneup is one every webmaster has heard of: denormalization. I had been using a view to select data from three tables with one call. The performance drag of that query was serious enough, though, that I’ve decided to complicate things a bit and copy the extra bits of data I need from the other tables into the main one for the query.

The speed gain is great, and, somewhat strangely, the denormalization actually cleaned up a bunch of my code. ErlyDB lacks a “one-to-one” relation, so it was impossible for me to say “each record in this view is really just a record in this other table with some extra data.” That made for a bit of hackery swinging from one type to another. Without that extra table, I think the code reads more clearly.

(Disclaimer: I’m far from being an relational database master, so it’s likely that there is a much better way to express everything I’m doing. But, I’m happy to be making what seems to be forward progress.)

The other main change is more Erlang-centric. Until now, I had been tracking sessions using a customization of the Yaws recommended session server. This is basically a central process that stores opaque data associated with an id string. Whenever your app gets a request, it pulls the cookie value out and checks with this central process to find out if there is any opaque data associated with this key. It works (quite well, in fact), but it seems like a bit of a bottle neck.

So, I’ve decided that there’s a more Erlangy way to do things. What BeerRiot is doing now is starting up a new process for each session, and saving that process id in a client cookie. Then, whenever a request comes in, if it has a cookie with a PID, we can try to contact that session’s handling process directly. No central service required.

It turns out that there’s loads of benefits to having this session hanging around beyond relieving the central service bottleneck. It can cache data, smartly (i.e. listen for updates, etc.). It’s a natural place to run background processes (like propagating live changes to durable storage). I see other potential uses, but since I haven’t tested them yet, I’ll hold my tongue to avoid getting too many hopes up. 😉

For Facebook developers: This process-session system wasn’t possible until just a few weeks ago, when Facebook started supporting cookies on the canvas page. Unfortunately, they only support them for canvas requests, and not for their “mock ajax.” For mock ajax, I’ve decided to just encode the cookie values in post parameters. It works (and it’s no more inconsistent than the rest of the Facebook Developer experience).

Update 2.Jan 18:52 EDT: If you spent any part of today poking at BeerRiot to see how the speed-ups turned out, you were probably rather dissatisfied. I just figured out that I didn’t fully rollout the update. 😛 It’s there now, and I think you’ll be much more impressed.

Erlang2Facebook Updates

I’ve just committed a couple of minor updates to the erlang2facebook library that I’m sure some of you are interested in.

The first (SVN revisions 7 & 9) is an API update to follow the Facebook team’s changes to profile_setFBML. Now, instead of just passing a single chunk of FBML, containing markup for the profile box, profile actions, and mobile profile, there are three distinct fields to shove those chunks in. Sorry about the non-consecutive SVN commits. 😛

The second update (SVN revision 8 ) is intended to show how to use ErlTL better (thanks for the tips, Yariv!). I’ve created, and moved all of the render_* functions from canvas_controller into it. This allows me to use the more HTML-like syntax (code efficiency), while also taking advantage of ErlTL’s automatic use of Erlang’s binaries (runtime efficiency).

RSS Feeds

In case you’re using this blog’s feed to get updates about new features on BeerRiot, here’s a notice that BeerRiot now has its own feeds. 🙂

I’m pretty green to the RSS scene, though, so if I’ve committed some terrible crime against RSS, please let me know, and I’ll get on fixing it. Or, if there’s just some way that I could improve the experience, which all of you RSS-fiends know about, those suggestions are most welcome.

BeerRiot Forums

Yes, the update I promised earlier in the week is now live. BeerRiot’s Forums are now open for discussion.

I know what some of you are thinking. “Forums?! Every site under the sun has forums. What did you waste weeks on forums for?” Clearly, if there was an ounce of sense in my head, I would have grabbed the nearest PhpBB implementation, tossed it up, and been done with it.

Well, I agree.

Except that I just couldn’t bear having such a disconnected, tacked on hunk of lameness. Even assuming that people wouldn’t mind the pain in the neck of having a second logon for a forum system, what I really wanted was a system that could really relate to the rest of the content on the site.

For example, I expect people to want to talk about a group (or several) of beers. Maybe someone will have a question about how two styles relate (porter and stout, hefeweizen and wheat, …), or they will have just returned from an event with an interesting selection. I think they should be able to easily set up a discussion about exactly the set of beers they want. So, since we were already doing this to some extent with tags, I have made it a key feature of discussion topics that they should be able to have tags associated with them.

I also expect people to want to get the word out about upcoming beer events. They’ll want to tell people where, and when, but then also discuss how it went afterward. So, in the events forum, all topics are allowed to have dates and locations associated with them.

As such, I went on my merry way developing. An opportunity to sling more Erlang code – how could I resist? 😉

As if that weren’t bad enough, there was the realization late in the project that the new forum system was so much better than the beer comment system that there was no way I could leave the elder in place. So, out came the old, and in went the new. And hey – I always wanted to let people comment directly on breweries, why not add that in too?

If you’ve been anywhere near the software industry (or, in fact, I suspect any industry), you recognize now the project spiral. “Just one more thing, then it will be ready.” That’s how it went for the first couple of weeks. Luckily I recognized the spiral at that point too, and cut off some of the broader plans to get this pushed out (don’t worry – they’re still on the agenda, just on a different deadline).

Believe it or not, I believe I was lucky enough to be able to pick joints at which to truncate plans, that after this big, bulky update I should be able to roll out some nice, smaller updates fairly quickly and smoothly. The next news item shouldn’t be three weeks away. 😉

So that’s that. In case you’re wondering, no, I still don’t regret choosing Erlang/ErlyWeb for this project. I didn’t run up against anything in this development that this system was incapable of handling. As mentioned above, I likely would have been delving deep into the guts of any forum system I chose, so development time would have been just as long. Getting to do it in a language that doesn’t annoy me [yet] was a real treat.

Despite reports to the contrary…

…I’m still alive and plugging away at BeerRiot code. I thought you all might just like to hear that despite the fact that I haven’t made a post here in a while, or updated the news section on the site, development is far from dead. There are things in the works, and I hope they will be ready soon.

But, I’m running into the oldest code-project slow-down in the book.[1] New code is making me rethink old code. Especially since I was much more of an Erlang amateur at the start of this project, my old code often looks ugly now. So, since my new code is doing very similar things to some of the oldest code here, I’m reworking the old stuff – mainly to avoid duplication, of course, but also because I understand the domain better now.

Unfortunately for you, the user, I’m still having quite a fun time slinging Erlang around, so it’s hard for me to just say, “Enough – post it!” But, I think I’ve defined the final goals, and the web of changes is starting to contract. The updates should happen soon (probably within a week), and then there will be lots of new toys to play with. 🙂

[1] Okay, so actually, there are older slowdowns in the book, but I ran into them too (that is, real life has demanded extra attention recently). But I think there’s a lull in them for a while, so things should progess faster now.

MySQL Death

Just a quick note this evening: BeerRiot was down from about 3:30am to 8am this morning. The MySQL server shut down. I restarted it as soon as I found it down.

Interesting: The MySQL server shut down, but the ErlyDB connection to it was still alive and waiting. As soon as the MySQL server came back up, ErlyDB automatically reconnected, and the site was live.

Sounds like as good of a reason as any to consider switching to Mnesia. I have to assume it’s easier to monitor that than to watch/restart MySQL.

Two Birds, One Hot Code Load

I bet there are a lot of people still questioning the utility of hot code loading. Especially in the web app field, it can seem a little gratuitous. PHP apps don’t need any special hot load facility – the script just gets reread from disk every once in a while.

Well, even if we ignore that there are likely parts of web apps that do need to run all the time, and are not just executed at request time, there’s still the web server to think about. And, guess what I did last week.

Yaws provides lots of nice utility functions. One in particular is yaws_api:htmlize/1, which takes an IoList as an argument, and returns the “same” list with the four big offenders (ampersand, double-quote, less-than, and greater-than) replaced with their HTML entities (& et al.). This function does exactly what you need when serving HTML directly to a modern web browser.

Unfortuntately, htmlize/1 doesn’t work perfectly when sending “FBML” to Facebook. During Facebook’s translation, it converts all characters with ASCII values greater than 127 to unrecognizeable characters, which come out as some form of “?” in a browser.

The fix is simple – just HTML-encode all characters over 127 as HTML entities of the form &#X;, where X is the decimal representation of the ASCII value. In Yaws 1.68, just add the following two lines just before yaws_api.erl:590 (the line with the guard for integer(X)):

htmlize_l([X|Tail], Acc) when integer(X), X > 127 ->
    htmlize_l(Tail, [$;, integer_to_list(X), $#, $&|Acc]);

Compile the new code by running make in the base directory of the Yaws source. If you’re running Yaws from a directory other than the source directory, copy ebin/yaws_api.beam to that other ebin directory.

Edit: See dbt’s comment for a way to skip the next paragraph in the simple case.

Now for the magic. My prefered way to load new code is to open a console to the web server’s erlang node. First, run “erl -sname Name”, where Name is any name other than that of your webserver. Once erl starts up, type C-g (control-g, for you non-Emacs-ers). You’ll be asked for a “User switch command”. Typing “h” will get you help here, but what you actually want to do is type “r yaws@host”, where “yaws@host” is the node name of the webserver’s erlang node. Typing “j” should now show two shell sessions. Connect to the second one with “c 2”.

Now that you’re connected to your web server’s erlang node, just type “l(yaws_api)” to load the new code. Any module calling into the yaws_api module will now automatically use the new code. Meanwhile, any code that was in the middle of a yaws_api module call will finish the call with the old code.

Edit: You won’t need the next paragraph either, if you followed dbt’s instructions.

When you’re done mucking about (I know you’ve just spent the last half hour figuring out what other bits of your webserver you can touch from here), type C-g again, then kill the remote console with “k 2”. Connect back to your first console with “c 1”, then exit it in the normal manner.

So, voila, Á now comes out of htmlize/1 as Á. International beer names show up properly in Facebook, and (oh boy!) BeerRiot Local now works in IE6 (which couldn’t parse those letters from the XML tag file). Two birds, one stone, I love it.

On a couple of side notes:

Thanks for the versioning system suggestions. I’ve settled on Mercurial for now, and I’m quite happy with it so far. Bit of a pain upgrading Python versions, but I probably should have done that long ago anyway.

And, I’ve been doing more than just building a website around beer this summer. I’ve also been growing my own ingredients! I made my first hop harvest earlier this week. It was only an ounce wet, which turned into about 1/8 oz. dry, but I was proud to have some success anyway. Here’s the proof:

Cascade hops growing in my back yard

Facebook Development (in Erlang)

As promised, here’s the post about the development process of the BeerRiot Facebook app.

Aside: Wow, rereading that last post about the Walled Gardens, I can’t believe I posted it. Totally fluffy lameness. Forgive me – this post will not be as bad.

I could very easily make this another post like others out there right now. Many people are upset about how unfinished the Facebook API is. Documentation is lacking (and only available online), specifications are weak, and test environments could be more feature-rich.

But, except for that little digression above, I promise, this is not one of those posts.

Instead, I’d really like to remark about how great it is developing this app in Erlang and Erlyweb!

There are nearly a dozen different Facebook API implementations – every object-oriented web language you can name. Facebook’s own official one is PHP5. I could have used the PHP version – Yaws comes complete with the ability to run PHP scripts. But, since the rest of BeerRiot is already in Erlyweb…

I rewrote the Facebook client in Erlang. And truthfully, it really wasn’t that bad at all. Erlang comes with very simple HTTP-communciation and XML-munging code. Hell, there’s even a simple way to compute an md5 sum. Once I figured out how to actually put together a proper POST and walk a rather verbose XML tree, the rest was just mimicking as closely as possible the official PHP scripts.

All this is not to say the process wasn’t without difficulties. Yaws’ standard url_encode proceedure doesn’t like nested io_lists very much, so I’m doing some ugly string/binary flattening. And, Facebook’s errors are nearly meaningless … but this isn’t about Facebook complaints. 😉

There are yet more benefits. Rather than dealing with cron firing up processes periodically, I have fully-supervised Erlang gen_servers up to date constantly with what data needs to be pushed out to Facebook. I can log into the running server and examine the current active sessions, monitor profile data pushes, etc.

Extending my existing Erlyweb application to handle new Facebook requests from the canvas page was even pretty simple (once I finally – I think – understood the Facebook session rules). Another controller, another view – bam!

One of my few real pains was source control. I may need to finally bite the bullet and leave CVS. I’m hearing good things about SVN. But that’s another topic.

So, chalk up one more win for Erlang/Erlyweb. Fantastic existing libraries. Quick development environment. Crazy server flexibility. I’m staying as long as I can.