Erlang at Basho

Published Thursday, September 25, 2008 by Bryan

Hmm...so, six-ish months ago, I posted that I had just spent a month working at a new job, where I got to code Erlang all day, and that I had a crazy month coming up, which would mean less attention here. That 'month' extended itself through the summer, and is going to continue through the fall, but I wanted to give you all an update.

Of course, anyone who really wanted to dig has already found that my new all-Erlang-all-the-time job is at Basho Technologies, but here's an announcement in a little more obvious place. We're creating a web-based system for salespersons and sales organizations to track, analyze, and improve their process. We are not just another CRM, and do not intend to become one. We're providing real tracking, guidance, and analysis, not just an 'online rolodex'.

But, enough about the business, you all want to know about the tech we're using, right? Okay, so it's not quite all-Erlang-all-the-time. Depending on the week, I spend anywhere from 40-90% of my time in Erlang, but because we're providing a web service, I spend the rest of the day in HTML (or our template/rendering language of choice), Javascript, or CSS. There may be a time at which I'll have to add Java and/or Flash to that mix as well, but for now, they're staying down the hall.

Erlang provides the logic for basically all of our server-side code. The webserver, mochiweb with a nice logic layer overtop code-named "webmachine" (which you'll probably hear more about in the coming months), is entirely coded in Erlang, and accepts Erlang modules as "resource providers." The storage system uses components from the distributerl project to provide a distributed storage solution. Even our template/rendering language is interpreted by Erlang code.

At this point, I'm sure there are a few of you with your jaws hanging open, minds full of protests and questions. "What about the lack of strings?" "What about the syntax?" I tell you now: these are not problems we've stumbled over.

"Lack of strings" is a complete misnomer. There are lists, iolists, and binaries. Through combinations of the three, we get everything we need. We do what coders have been doing for decades: we wrap tricky, repetitive, often-needed code in libraries and move on. Honestly, we're having more trouble with the standard datetime tuple (no standard for timezone marking, confusing timezone expectations of conversion functions).

To truly get an idea of how much trouble the syntax is, I recommend you try this excercise: code in Erlang-only for a few weeks, then try coding in something else, like Javascript. I think you'll realize that each language has its own bunch of syntax quirks.

We were able to run the exercise above in our first few weeks of coding. Many of us had never touched Erlang, and some had never touched Javascript. Each language took a couple of days for the learner to be able to produce something useful, and another week or so for the same learner to be able to do it on their own in a reasonable amount of time. We do have complaints about the way each language is written now, but they share little with the complaints we had when starting out.

Case in point: records. Everyone's favorite structure to beat up (after strings, of course). At first they seem like overly-verbose, half-baked forms of structures from other languages. Then comes the epiphany: they're not designed to take the place of those structures. The best way to think about records is to envision C-structs full of void pointers. Then you can realize that their real power is in pattern matching - function guards, selective extraction, format assertions.

But just describing the ways in which Erlang hasn't hindered us isn't very impressive - how has Erlang actually helped us? Well, we've been able to implement our own home-grown distributed storage system in just a couple of months. We've also been able to create an entirely new design of webserver that makes writing truly RESTful webservices simple and quick. We've been able to troubleshoot problems by quickly opening shell connections to the running webservers and learning exactly what is going on. We've written lots of extra self-contained services that can export, import, analyze, and otherwise munge live data easily. In short, we've been able to iterate quickly over lots of proposed solutions to problems and choose based on experience, rather than weak assumptions and hearsay.

Or, maybe it's just that I work with an amazing team that's able to roll with punches quickly, and usually end up with a better solution in the end than the one we designed in the beginning. Could be - I know this team is the most talented I've met since college. :)

Sitting here almost eight months since starting Erlang full-time, and almost two years since picking it up for personal projects, I'd say that I'm here to stay for a while yet. I doubt neither that there will be another language in the future that will steal my attention, nor that there are other languages I'd choose for other types of projects. But, for distributed web-based services today, Erlang has my vote.

Categories: Erlang