Archive for March, 2009|Monthly archive page

HTTP Decision Graph Comes to Life

Sometime last year, a flowchart describing the processing of an http request (as rendered by Alan Dean) made its way around the net.

http-graph-small-unmarked

I thought, “Wouldn’t it be cool if your webserver could plot the path it took on that chart in handling a request?”

http-graph-marked-200

Well, now it can – if you’re using the latest Webmachine. The above graph is scaled down from a tool I created that is packaged with the just-released Webmachine 1.0.

You can read all about the tool on the Webmachine Debugging page, but the basic idea is: draw an edge between each node in the graph that is traversed during the processing of a request. Clicking on decisions brings up a panel that details what resource module functions were called at that decision (as well as what their parameters and return values were). There are also panels for the details about the request itself (method, path, headers, body) and the response (code, headers, body).

I’ve put up three example traces from BeerRiot’s beer_resource:

  1. The first hits /beer/1, and runs successfully all the way to 200: trace-example-200.html
  2. The second hits /beer/9999, which doesn’t exist, and runs to 404: trace-example-404.html
  3. The third hits /beer/536, which used to exist, but has since been merged with beer 202, so it runs to 301: trace-example-301.html

You’re floored, right? You just can’t wait to get your hands on it, right? </modesty> Well, doing so is easy. Once you have your Webmachine resource written, just change your init/0 function from:

init(Config) ->
   {ok, Config}.

to:

init(Config) ->
   {{trace, "/tmp"}, Config}.

Recompile and reload the module, then issue an HTTP request to it. You should see a file show up in /tmp with the extension .wmtrace.

Now open the Erlang shell for your webmachine application and type:

wmtrace_resource:add_dispatch_rule("wmtrace", "/tmp").

Tab over to your browser and hit /wmtrace/. You should see the list of .wmtrace files in your /tmp directory. Click on one of them, and you’ll be in the trace inspection utility.

For any pre-webmachine-1.0 users, getting access to this utility requires converting your resources to the new webmachine with referential transparency, but I can tell you from experience that that process is largely mechanical, and not that time consuming. I translated BeerRiot’s ~7000loc in about 2 hours (including testing).

I’d love to hear feedback about the trace tool. It’s the product of about three days of hacking (one proof-of-concept, one nearly-complete-rewrite, one actual-improvement), so I’m not hard-set on much of anything.

‘This’ is not what you think it is

It took me a little while to get used to Erlang. It was a couple of days before I knew the syntax without looking at examples. Several more days before I knew what utility functions to expect, and what their parameters should look like. A month or two before gen_server and gen_supervisor clicked.

Another language has taken me longer to learn, though: Javascript. Sure, it was easy to just write some C-esque, Java-like syntax for simple calculations. Objects were pretty straightforward, as long as I didn’t do anything complicated. But when it got to lots of event handling, I found myself nervously copying the few examples that I knew worked. It took a thorough reading of Javascript: The Definitive Guide, and several intensive weeks of JS-coding for it to sink in.

I now blame my lost hours of Javascript paranoia on one word: this.

Javascript’s this is completely different from the this keyword of Java and C++ (the other two languages I know well enough to speak about that also have this keywords). Where Java’s this is the object in which the function is instantiated, Javascript’s this is the “execution context.”

For simple calls, like a.f(); Java’s and Javascript’s definitions are pretty much the same thing: this == a. In Javascript, though, you can say var q = a.f; q();, and suddenly this is not a, but rather the global context. To get this set to a again, you have to use q.apply(a);.

Even better – here’s a quick example:

js> var x = "global";
js> var f = function() { print("f: "+this.x); };
js> var a = {x:"object-a", f:f};
js> var b = {x:"object-b", f:function() { print("b.f: "+this.x); }};

js> f();
f: global

js> a.f();
f: object-a

js> b.f();
b.f: object-b

js> f.apply(a);
f: object-a

js> a.f.apply(b);
f: object-b

js> b.f.apply(a);
b.f: object-a

js> var q = b.f;
js> q();
b.f: global

If you’re as unfamiliar with this as I was, it’s best if you reason for yourself why each line prints what it does. The basic idea is that the output “function:context” tells you what function is executing (“f” or “b.f”), and in which context (“global”, “object-a”, or “object-b”).

If you’ve followed the example successfully, you can see that any function can be executed in such a way that this is whatever you want it to be. It doesn’t matter if the function is defined “in” some other object; if it can be referenced, it can be applied to the object of your choice.

I woke up to the operation of this in a backward way: I first learned that Javascript was lexically scoped. That’s right, just like other great languages, as soon as you type function() { }, you “draw a double-bubble”, and every time you execute it, you “drop a box”. More to the point, Javascript supports closures, just like Scheme*!

“So, if Javascript has closures, what’s this for?” I asked myself. It seemed like a strange loophole – a way to define scope in direct conflict with the rest of the language. I shunned this. I didn’t need it. I knew how to operate in syntactic scope.

It wasn’t until I dug back into my event-handling code that it dawned on me: it’s the interaction between the two “context” systems that makes things interesting. Most Javascript event code (from DOM to Google Maps) executes the listener/handler functions with this set to the object that triggered the event. That’s right – you get references both to the context in which the function was defined (implicitly through scope) and to the context in which the function should be applied (explicitly through this).

For example, try out this code (assuming you have both jQuery and firebug installed):

Update: I’ve put up a version of this code here which puts the “console” on the page, so you don’t need firebug or your own jQuery.

<html>
  <head>
    <script type="text/javascript" src="jquery-1.2.6.js"></script>
    <script type="text/javascript">
function logClick(bc, bn, gc, gn) {
   console.log(["You've clicked",bc,bn,"times",
                "- of",gc,"total for",gn].join(' '));
};

$(function() {
    $('div').each(function() {
       var groupCount = 0;
       $(this).find('button').each(function() {
          var buttonCount = 0;
          $(this).click(function() {
             logClick((++buttonCount),
                      $(this).text(),
                      (++groupCount),
                      $(this).parent().find('div').text());
          });
       });
    });
 });
    </script>
  </head>
  <body>
    <div>
      <div>group a</div>
      <button>foo</button>
      <button>bar</button>
    </div>
    <div>
      <div>group b</div>
      <button>zot</button>
      <button>qux</button>
    </div>
  </body>
</html>

What you get is two “groups”, with two buttons each. Each time you click a button, the name of the button, as well as the number of times it has been clicked are printed on the console. Just to make it a little more interesting, the name of the group, as well as the total number of clicks of buttons in that group are also printed.

Two closures give us local storage for click counts: the outer one closes over groupCount, while the inner one closes over buttonCount (and also over groupCount by transitivity, of course). jQuery’s each function applies its argument to each element in the matched list, so this is set to each div in the outer iteration, and this is set to each button in the inner iteration. jQuery’s click function registers its argument in such a way that the handler gets applied to the element that triggered the event, so this is once again set to each button element in the inner-most function.

The end result is four functions registered as click handlers, each attached to a different button, each closing over its own buttonCount, and two pairs of those functions closing over the same groupCount. When the handler is called, this is set to the button, and we can extract text from the page for logging.

Yes, this is a trivial, contrived example that’s not good for much on its own, but this technique can be applied to any place a callback is used: iteration constructs and DOM events (as shown above), XHRs, Google Maps events, etc. Need to share a reference between several callbacks (or several executions of the same callback), but don’t want to pollute your namespace? Closures are the answer. Need to know why your callback is being called? this to the rescue.

A coworker and I were discussing all of this the other day, when the question arose, “Why a magical this? Why not just use the calling convention that ‘this’ is the first parameter of the function?” Well, I’m still trying to come up with a good answer for that one. In truth, many libraries do also pass an argument to the callback function that is equal to this. My strongest feelings so far: standardization and syntactic sugar.

By standardizing that this always means this very specific thing, we dodged the bullet of some libraries supporting it, while others don’t, while still others call it something else. Sometimes you just have to make a decision to ensure that it has been decided.

Without this defined as-is, the syntax for “applying foo’s bar member to foo” becomes foo.bar(foo); why name foo twice? In addition, all function signatures now become function bar(this, x, y, z, ...), whether they’re interested in this or not.

As I dive into playing more with prototypes, I expect to find additional interesting qualities to the use of this, but as yet, I can’t speak to them.

I’m sure this has all been written before (I still haven’t read all of the Douglas Crockford text that has been recommended to me), but I was just so excited to find that Javascript was a more interesting language than I originally thought, that I had to write about it. Maybe if enough of us say it, we can prevent future programmers from wasting their time on the same misconceptions.

* “Scheme is like Victoria’s Secret: It’s supposed to be elegant, but really it’s just dirty,” is printed next to my picture in my yearbook. I’ve since married the woman who said it, and learned the error of my ways. Likely not related events, but who can say for sure?

Follow

Get every new post delivered to your Inbox.