Archive for the ‘Webmachine’ Category

Roundtripping the HTTP Flowchart

Webmachine hackers are familiar with a certain flowchart representing the decisions made during the processing of an HTTP request. Webmachine was designed as a practical executable form of that flowchart.

It has long bugged many of the Webmachine hackers that this relationship is one-way, though. Webmachine was made from the graph, but the graph wasn’t made from Webmachine. I decided to change that in my evenings last week, while trying to take my mind off of Riak 1.0 testing.

This is a version of the HTTP flowchart that only a Webmachine hacker could love. It’s ugly and missing some information, but the important part is that it’s generated by parsing webmachine_decision_core.erl.

I’ve shared the code for generating this image in the gen-graph branch of my webmachine fork. Make sure you have Graphviz installed, then checkout that branch and run make graph && open docs/wdc_graph.png.

In addition to the PNG, you’ll also find a docs/wdc_graph.dot if you prefer to render to some other format.

If you’d really like to dig in, I suggest firing up an Erlang node and looking at the output of wdc_graph:parse("src/webmachine_decision_core.erl"):

[{v3b13, [ping],                     [v3b13b,503]},
 {v3b13b,[service_available],        [v3b12,503]},
 {v3b12, [known_methods],            [v3b11,501]},
 {v3b11, [uri_too_long],             [414,v3b10]},
 {v3b10, [allowed_methods,'RESPOND'],[v3b9,405]},
 {v3b9,  [malformed_request],        [400,v3b8]},
...

If you’ve looked through webmachine_decision_core at all, I think you’ll recognize what’s presented above: a list of tuples, each one representing the decision named by the first element, with the calls made to a resource module as the second element, and the possible outcomes as the third element. Call wdc_graph:dot/2 to convert those tuples to a DOT file.

There are a few holes in the generation. Some response codes are reached by decisions spread across the graph, causing long arrows to cross confusingly. The edges between decisions aren’t labeled with the criteria for following them. Some resource calls are left out (like those made from webmachine_decision_core:respond/1 and the response body producers and encoders). It’s good to have a nice list for future tinkering.

Riak Presented at NYC NoSQL – slides, text & video

I had the pleasure of attending the NYC NoSQL Fall ’09 Meetup/Mini-Conference last Monday. Great talks, all around. I thought it was a good mix of use-case analysis and technology introduction.

In addition to enjoying everyone else’s presentations, I also presented Riak. It was a quick 12-minute talk, followed by 2.5 minutes of questions, but the response I got was great. People really dug in and had interesting observations and questions to discuss afterward.

If you weren’t able to make the event, Brendan has posted video of my talk. I have also posted an HTML slides-and-text version of my talk, if you prefer reading over watching and listening.

Riak Demo: Stickynotes

Warning: This blog post is woefully out of date:

  1. Instead of building Riak from source, you should download a pre-compiled release from downloads.basho.com (though the source is still available, if you want).
  2. Starting Riak is now done with the bin/riak start command, not start-fresh.sh Stopping is also easily done with bin/riak stop
  3. Jiak is no longer the preferred HTTP interface. There is now one that accepts any content type, not just JSON.
  4. Without Jiak, back-linking of notes will need to be done with post-commit hooks.

There are probably other things I’ve overlooked. Really, you should just head over to the Riak wiki to get the intro.


Basho released Riak to the world just over a week ago. Some think that the docs are good enough that further explanation is not necessary. But, if you’re looking for a little more introduction, read on.

The short version of this post: Riak is as simple as downloading it and hitting the HTTP interface:

1$ hg clone http://bitbucket.org/basho/riak/
2$ cd riak
3$ make
4$ ./start-fresh.sh config/riak.erlenv

5$ curl -X PUT -H "Content-type: application/json" \
5>   http://localhost:8098/jiak/foo \
5>   -d "{\"schema\":{\"allowed_fields\":[\"bar\"],\"required_fields\":[\"bar\"],\"read_mask\":[\"bar\"],\"write_mask\":[\"bar\"]}}"

6$ curl -X PUT -H "Content-type: application/json" \
6>   http://localhost:8098/jiak/foo/baz \
6>   -d "{\"bucket\":\"foo\",\"key\":\"baz\",\"object\":{\"bar\":\"Hello World\"},\"links\":[]}"

7$ curl http://localhost:8098/jiak/foo
{"schema":{"allowed_fields":["bar"],"required_fields":["bar"],"read_mask":["bar"],"write_mask":["bar"]},"keys":["baz"]}

8$ curl http://localhost:8098/jiak/foo/baz
{"object":{"bar":"Hello World"},"vclock":"MzIwsDSwMDQyMjAyNjXQLcpMzHYwNDLXMwBCQ3SusYGZoZGxvqG+mbGJobmxkbmFsQEA","lastmod":"Wed, 12 Aug 2009 20:23:50 GMT","vtag":"6DxaqiRCDBevf03tzPZpzl","bucket":"foo","key":"baz","links":[]}

Commands 1-4 download and start Riak. Command 5 creates the bucket foo, and tells Jiak, Riak’s HTTP interface, that objects in bucket foo must have a bar field that is readable and writable. Command 6 creates the object baz in bucket foo. Command 7 gets the listing of bucket foo. Command 8 retrieves object baz in bucket foo.

Aside: Some have found it difficult to stop Riak, so I’ll throw the tip out here: killall heart. You may also have to kill the erl command afterward, but if you don’t kill heart first, Riak will just come right back.

That’s all it takes to use Riak. You can stop reading now and happily speak REST to it for the rest of your application’s lifetime.

If you’re still hanging around, though, maybe you’d be interested in a few more features that Jiak has to offer, like link-walking and field validation. To demonstrate those, I’ll spend the rest of this post describing the creation of an application on Riak.

Let’s say that I wanted to create a note-taking system, something like the fine Erlang/Mochiweb demo put together by the guys at Beebole.

There will be two kinds of objects in my system: notes and groups. The properties of a note will be the text of the note, the color, the position on the screen, and the stacking order. The properties of a group will be the name of the group and the list of notes in the group.

I’ll start by whipping up two modules to manage those buckets for me. These modules tell jiak_resource how to validate the structure of objects in these buckets (jiak_resource is the Webmachine resource that runs Riak’s HTTP interface). They’ll follow the basic structure of jiak_example.erl, which ships with Riak.

-module(groups).
-export([init/2, auth_ok/3, bucket_listable/0, allowed_fields/0,
         required_fields/0, read_mask/0, write_mask/0,
         expires_in_seconds/3, check_write/4, effect_write/4,
         after_write/4, merge_siblings/1]).

init(_Key, Context) -> {ok, Context}.

auth_ok(_Key, ReqData, Context) -> {true, ReqData, Context}.

bucket_listable() -> true.

allowed_fields()  -> [<<"name">>].
required_fields() -> allowed_fields().
read_mask()       -> allowed_fields().
write_mask()      -> allowed_fields().

expires_in_seconds(_Key, ReqData, Context) ->
    {600, ReqData, Context}.

check_write({_PutType, _Key}, JiakObject, ReqData, Context) ->
    {ObjDiffs,_} = Context:diff(),
    case lists:foldl(fun check_diff/2, [], ObjDiffs) of
        [] ->
            {{ok, JiakObject}, ReqData, Context};
        Errors ->
            {{error, list_to_binary(string:join(Errors, ", "))},
             ReqData, Context}
    end.

check_diff({<<"name">>, _, Value}, ErrorAcc) ->
    if is_binary(Value) -> ErrorAcc;
       true             -> ["name field must be a string"|ErrorAcc]
    end.

effect_write(_Key, JiakObject, ReqData, Context) ->
    {{ok, JiakObject}, ReqData, Context}.

after_write(_Key, _JiakObject, ReqData, Context) ->
    {ok, ReqData, Context}.

merge_siblings(Siblings) ->
    jiak:standard_sibling_merge(Siblings).

The groups module ensures that every group has a name field (through implementation of allowed_fields/0, required_fields/0, read_fields/0, and write_fields/0), and that the value of that field is a string (using check_write/4 and check_diff/2).

-module(notes).
-export([init/2, auth_ok/3, bucket_listable/0, allowed_fields/0,
         required_fields/0, read_mask/0, write_mask/0,
         expires_in_seconds/3, check_write/4, effect_write/4,
         after_write/4, merge_siblings/1]).

init(_Key, Context) -> {ok, Context}.

auth_ok(_Key, ReqData, Context) -> {true, ReqData, Context}.

bucket_listable() -> true.

allowed_fields() ->
    [<<"text">>, <<"x">>, <<"y">>, <<"z">>, <<"color">>].

required_fields() -> allowed_fields().
read_mask()       -> allowed_fields().
write_mask()      -> allowed_fields().

expires_in_seconds(_Key, ReqData, Context) ->
    {600, ReqData, Context}.

check_write({_PutType, Key}, JiakObject, ReqData, Context) ->
    {ObjDiffs,_} = Context:diff(),
    case lists:foldl(fun check_diff/2, [], ObjDiffs) of
        [] ->
            {{ok, JiakObject}, ReqData, Context:set_prop(key, Key)};
        Errors ->
            {{error, list_to_binary(string:join(Errors, ", "))},
             ReqData, Context}
    end.

-define(COLORS, [<<"yellow">>, <<"pink">>, <<"green">>, <<"blue">>]).

check_diff({<<"text">>, _, Value}, ErrorAcc) ->
    if is_binary(Value) -> ErrorAcc;
       true             -> ["text field must be a string"|ErrorAcc]
    end;
check_diff({Coord, _, Value}, ErrorAcc)
  when Coord==<<"x">>;Coord==<<"y">>;Coord==<<"z">> ->
    if is_integer(Value) -> ErrorAcc;
       true ->
            [io_lib:format("~s field must be an integer", [Coord])
             |ErrorAcc]
    end;
check_diff({<<"color">>, _, Value}, ErrorAcc) ->
    case lists:member(Value, ?COLORS) of
        true -> ErrorAcc;
        false ->
            [io_lib:format("color field must be one of (~s)",
                           [string:join([binary_to_list(C)||C<-?COLORS],
                                        ",")])
             |ErrorAcc]
    end.

effect_write(_Key, JiakObject, ReqData, Context) ->
    {{ok, JiakObject}, ReqData, Context}.

after_write(_Key, JiakObject, ReqData, Context) ->
    spawn(fun() ->
                  [[_, GroupKey, _]] = jiak_object:links(JiakObject, groups),
                  {ok, C} = jiak:local_client(),
                  {ok, G} = C:get(groups, GroupKey, 2),
                  Key = Context:get_prop(key),
                  C:put(jiak_object:add_link(G, notes, Key, <<"note">>), 2)
          end),
    {ok, ReqData, Context}.

merge_siblings(Siblings) ->
    jiak:standard_sibling_merge(Siblings).

The notes module ensures that all notes have a text field that is a string; x, y, and z fields that are integers; and a color field that is one of a specific list of strings (using the same functions as the groups module used).

One more interesting thing happens in the notes module. If you look at after_write/4, you’ll see that it fetches the groups object that the note links to, and adds the note to that group’s links. Jiak calls after_write/4 after the note has been stored in Riak, so what I’ve written here is effectively an automatic back-link monitor. We’ll return to the concept of links in a moment.

If I put the notes and groups modules in place, I can fire up Riak and immediately being sending HTTP requests to its Jiak interface to create and modify groups and notes. For example:

$ curl -X PUT -H "Content-type: application/json" \
>   http://127.0.0.1:8098/jiak/groups/todos \
>   -d "{\"bucket\":\"groups\",\"key\":\"todos\",\"object\":{\"name\":\"todo\"}\"links\":[]}"

$ curl -X PUT -H "Content-type: application/json" \
>   http://127.0.0.1:8098/jiak/notes/blog \
>   -d "{\"bucket\":\"notes\",\"key\":\"blog\",\"object\":{\"text\":\"finish blog post\",\"x\":0,\"y\":0,\"z\":0,\"color\":\"green\"},\"links\":[[\"groups\",\"todos\",\"open\"]]"

These two lines would create a group named todo with a note labeled finish blog post.

Now, about those links. See the ["groups","todos","open"] item in the links field of that notes object? That’s a link to the groups object named todos, and I’ve tagged it open. The affect_write/4 function in the notes module will add a link to the groups object of the form ["notes","blog","note"].

What does this get me? More than just record-keeping: I can now use the Jiak utility jaywalker to get all of the notes in the todo group with a single query:

$ curl http://127.0.0.1:8098/jiak/groups/todos/notes,_,_

You’ll recognize the first part, through /jiak/groups/todos/. The segment after that is a link query. This ones says "all notes objects, with any tag." The links are structured as {bucket},{tag},{accumulate} segments, with underscore meaning "any."

The example query will return an object with a results field that is a list of lists of results. That is, if I were to store the data returned from that query in a variable called data, my list of notes would be at data.results[0].

I’m not limited to one hop, either. If there were also person objects in my system, linked from notes objects as authors I might get all of the authors of the notes in a group with:

$ curl http://127.0.0.1:8098/jiak/groups/todos/notes,_,_/person,author,_

More information on link-walking can be found in the jaywalker_resource documentation.

The curl commands and the HTTP requests they represent are pretty simple, but I’m going to be hitting these resources from Javascript running in a browser. So, I’ll wrap it all up in a nice utility class:

function JiakClient(BaseUrl, Opts) {
    this.baseurl = BaseUrl;
    if (!(this.baseurl.slice(-1) == '/'))
        this.baseurl += '/';

    this.opts = Opts||{};
}

JiakClient.prototype.store = function(Object, Callback, NoReturnBody) {
    var req = {
        contentType: "application/json",
        dataType: "json"
    };

    if (this.opts.alwaysPost || !Object.key)
        req.type = 'POST';
    else
        req.type = 'PUT';
    
    req.url = this.baseurl+Object.bucket+'/';
    if (Object.key) req.url += Object.key;
    
    if (!(this.opts.noReturnBody || NoReturnBody))
        req.url += '?returnbody=true';

    if (typeof Callback == 'function')
        req.success = Callback;

    req.data = JSON.stringify(Object);

    return $.ajax(req);
}

JiakClient.prototype.fetch = function(Bucket, Key, Callback) {
    return $.ajax({
        url:      this.baseurl+Bucket+'/'+Key,
        dataType: "json",
        success:  Callback
    });
}

JiakClient.prototype.remove = function(Bucket, Key, Callback) {
    return $.ajax({
        type:    'DELETE',
        url:     this.baseurl+Bucket+'/'+Key,
        success: Callback
    });
}

JiakClient.prototype.walk = function(Start, Spec, Callback) {
    var req = {
        dataType: "json",
        success: Callback
    };

    if ('bucket' in Start)
        req.url = this.baseurl+Start.bucket+'/'+Start.key+'/';
    else
        req.url = this.baseurl+Start[0]+'/'+Start[1]+'/';

    for (i in Spec) {
        req.url += (Spec[i].bucket||'_')+','+
            (Spec[i].tag||'_')+','+
            ((Spec[i].acc || i == Spec.length-1) ? '1' : '_')+'/';
    }

    return $.ajax(req);
}

Now I can create the same effect as the curl commands with this code:

var J = new JiakClient("/jiak/");
J.store({
  bucket:"groups",
  key:"todos"
  object:{"name":"todo"}
  links:[]
},
function(group) {
  J.store({
    bucket:"notes",
    key:"blog",
    object:{
    },
    links:[["groups",group.key,"open"]]
  });
});

With the request for "all notes in the todo group" looking like:

J.walk({bucket:"groups", key:"todos"},
       [{bucket:"notes"}],
       function(data) {
         var notes = data.results[0];
         //do stuff...
       });

Okay, now I have my backend and frontend – I’d better fill in the middle. Any intermediary webserver could work, but because Riak comes with Webmachine included, I’ll just setup a quick Webmachine app.

$ cd $RIAK_HOME
$ deps/webmachine/scripts/new_webmachine.erl stickynotes ..
$ cp config/riak.erlenv ../stickynotes/riak-config.erlenv

I copied the Riak config (riak.erlenv) because there are two customizations I want to add to it:

{add_paths, ["../stickynotes/ebin"]}.
{jiak_buckets, [notes, groups]}.

The former puts the stickynotes ebin in Riak’s code path, so Jiak can reach the notes and groups modules I just wrote. The latter does nothing but force the atoms 'notes' and 'groups' into the Riak node so Jiak can use list_to_existing_atom/1.

To my new Webmachine app, I’ll add four things:

  1. the notes and groups modules I just wrote
  2. the static files from Beebole’s example applcation (HTML, CSS, JS) … modified a bit to load and use the jiak.js utility from above
  3. a simple static resource server
  4. a simple proxy resource to pass requests through to jiak (the couchdb_proxy from my wmexamples repo will do fine)

Once I have everything aligned (including dispatch setup correctly), I just start Riak and my Webmachine node:

$ cd $RIAK_HOME
$ ./start-fresh.sh ../stickynotes/riak-config.erlenv
$ cd ../stickynotes
$ ./start.sh

…then point my browser at http://localhost:8000/, and I see the app’s UI with an empty group ready to store some notes. I recommend opening Firebug to watch the requests fly by.

I realize that I’ve glossed over a bit of the Webmachine application stuff, but that’s because it’s mostly rehash of older posts. The better way to cover all of that material is for me to tell you to open up the demo/stickynotes directory in the Riak repo you just cloned, and read the code written there yourself. :)

Webmachine Bloggy Goodness

A whole bunch of people (no half-bunches in these parts) are picking up Webmachine and writing about their experiences. If you’re looking for a few more people to follow:

Andy Gross (a coworker) is writing about file uploads. Actual webmachine details to follow in his next post – this first one is about the client side of the equation. Update: that second post is up, and includes an example of the new “streamed body” feature in webmachine 1.3.

Seth Falcon is covering the design and implementation of a URL-shortening service. Again, webmachine details to follow, but there’s some good talk about using Mnesia for storage in the current post.

Marc Worrell is laying down the outline of his new CMS, Zophrenic. This looks like a very ambitious project, and I’m anxious to see its release.

Update: More blogs from the comments

Daniel Kwiecinski just covered redirecting (3xx responses), and also recently showed off his own static-file-serving resource. He also tips his hat to wmtrace, so he gets brownie points. ;)

Great reads, but don’t take my word for it.</burton>

Webmachine POST Example

Many people have asked for an example Webmachine resource that responds to POST. If you follow my twitter feed, you may have caught this gem.

I figured that example could use a little fleshing out, so I’ve added a resource to my wmexamples repo.

formjson_resourc.erl makes an attempt at demonstrating the simplest way to handle a POST, while also demonstrating the difference between content-producing functions (to_json/2 in this example, and others named in content_types_provided/2), which put content in the response body simply by returning it, and other functions, which have to put content in the response body by returning a modified ReqData.

For another example of handling POST, read demo_fs_resource.erl that comes with Webmachine. It implements post_is_create/2, create_path/2, content_types_accepted/2, and accept_content/2 to handle POST requests. (Incidentally, demo_fs_resource is a good example of many Webmachine resource functions.)

Updated to include content_types_accepted/2 in the list of functions handling POST requests – thanks for catching it, Lou!

Webmachine Slideshow Intro

Justin Sheehy has just posted his video slideshow introduction to webmachine. The video contains the slides and most of the talk that Justin gave at the Bay Area Erlang Factory last month. If you’ve been thinking about checking out Webmachine, this half hour is well worth your time.

Webmachine has also gained several new features recently, including delayed body receive and a more independent request dispatcher. Partial receive and send is even available in tip.

Unfortunately, the trace utility required changes that broke compatiblity with old trace files. Support for those old traces could probably be hacked in without too much trouble if you need it (the cause was mainly just a change to a record structure). I assumed that trace files are somewhat ephemeral, though – really only needed for debugging, and then thrown away.

Finally, if you’re looking for example resources and dispatch tables, I recommend checking out my wmexamples repo on bitbucket. It has a smattering of working resources that can hopefully give you a feel for how we normally write them.

CouchDB Proxy Webmachine Resource

Alright, here it is, proof that Webmachine and CouchDB can coexist peacefully: I’ve posted an example Webmachine resource that proxies requests back and forth to a CouchDB instance. (Thanks for the kick, benoitc.)

I dislike the way this blog renders my code, so I’ve decided to start posting my examples on bitbucket, wrapped in a simple Webmachine app, so they’re ready to run. The repo you want is wmexamples.

You’ll find the CouchDB proxy in src/couchdb_proxy.erl.

Watch that repo for other updates. I’ll be going back in the next few days and adding env_resource and some tracing stuff.

Simple Webmachine Extension (4/4): DELETE

This is the third post in a four-part series about extending a simple Webmachine resource. The first part discussed adding support for the HTTP method PUT, and the second part added basic authorization, while the third part added conditional requests through ETag.

DELETE

I can modify variables through PUT; how about removing them altogether with DELETE?

First a little sidetrack. While Erlang exposes os:getenv/1 and os:putenv/2, it does not expose an os:deleteenv/1. So, what I’m going to do instead is adopt the convention that a variable set to “nothing” (i.e. the empty list) has been deleted. This requires a little modification to resource_exists/2 to filter out variables with empty-list values. (See the full source at the end of this post.)

Sidetrack #2: I also noticed that there was a bug preventing values from containing the equals sign, so while hacking resource_exists/2, I fixed that as well.

With the empty-list-is-deleted convention in hand, I can implement DELETE handling like this:

-export([delete_resource/2]).

delete_resource(RD, Ctx) ->
    os:putenv(wrq:path_info(env, RD), []),
    {true, RD, Ctx}.

I only want to allow one variable to be deleted at a time, so I’ll modify allowed_methods/2 such that it only adds 'DELETE' to the method list if the request path is of the form /_env/VARIABLE.

I also want DELETE to require authorization, so I’ll modify is_authorized/2 to watch for both PUT and DELETE.

And that’s that. I can now use the following curl command to delete MY_VAR:

$ curl -u webmachine:rules -X DELETE http://localhost:8000/_env/MY_VAR

Wrap Up

Before I get to the complete source, I want to take a moment to highlight something I stressed in an earlier blog post: Did you notice that none of the code I’ve displayed in the last few days has mentioned specific HTTP response codes?

The env_resource now runs all over the HTTP decision flowchart, returning everything from 405 when methods other than GET, HEAD, PUT, and DELETE are issued (or just the first three in the whole-environment case), to 401 when PUT or DELETE is attempted without the proper credentials, and 412 when ETags don’t match on a PUT (or 304 when they do match on a GET!), and yet, I didn’t mention a single code. I simply described the properties I wanted the resource to have, and let Webmachine do the translation.

I stress this because I think it’s important to consider the power of an HTTP translation system. I’ve seen reduced development time as an effect of not having to worry about getting to the right response code in every corner case, while still adding necessary headers. I’ve also seen reduced troubleshooting time as an effect of being able to read through a resource that only concerns itself with describing its properties, rather than including a lot of mechanics for tearing apart HTTP requests and building up HTTP responses.

Plug

If this series has raised your interest level in Webmachine, I recommend you attend Justin Sheehy’s talk at the Bay Area Erlang Factory.

The Complete Code

%% dispatch:
%% {["_env"],      env_resource, []}.
%% {["_env", env], env_resource, []}.

-module(env_resource).
-export([init/1, content_types_provided/2, resource_exists/2, to_json/2]).
-export([allowed_methods/2, content_types_accepted/2, from_json/2]).
-export([is_authorized/2]).
-export([generate_etag/2]).
-export([delete_resource/2]).
-include_lib("webmachine/include/webmachine.hrl").

init(_) -> {ok, undefined}.

content_types_provided(RD, Ctx) ->
    {[{"application/json", to_json}], RD, Ctx}.

resource_exists(RD, Ctx) ->
    case wrq:path_info(env, RD) of
        undefined ->
            Result = [ {K, string:join(V, "=")}
                       || [K|V] <- [ string:tokens(E, "=")
                                     || E <- os:getenv() ],
                          V /= [] ],
            {true, RD, {struct, Result}};
        Env ->
            case os:getenv(Env) of
                false  -> {false, RD, Ctx};
                []     -> {false, RD, Ctx};
                Result -> {true, RD, Result}
            end
    end.

to_json(RD, Result) ->
    {mochijson:encode(Result), RD, Result}.


%% PUT support

allowed_methods(RD, Ctx) ->
    {['GET', 'HEAD', 'PUT'
      |case wrq:path_info(env, RD) of
          undefined -> [];
          _         -> ['DELETE']
       end],
     RD, Ctx}.

content_types_accepted(RD, Ctx) ->
    {[{"application/json", from_json}], RD, Ctx}.

from_json(RD, Ctx) ->
    case wrq:path_info(env, RD) of
        undefined ->
            {struct, MJ} = mochijson:decode(wrq:req_body(RD)),
            [ os:putenv(K, V) || {K, V} <- MJ ];
        Env ->
            MJ = mochijson:decode(wrq:req_body(RD)),
            os:putenv(Env, MJ)
    end,
    {true, RD, Ctx}.


%% AUTH support

-define(AUTH_HEAD, "Basic realm=MyOSEnv").

is_authorized(RD, Ctx) ->
    case wrq:method(RD) of
        PD when PD == 'PUT'; PD == 'DELETE' -> basic_auth(RD, Ctx);
        _                                   -> {true, RD, Ctx}
    end.

basic_auth(RD, Ctx) ->
    case wrq:get_req_header("Authorization", RD) of
        "Basic "++Base64 ->
            case string:tokens(base64:mime_decode_to_string(Base64), ":") of
                ["webmachine", "rules"] -> {true, RD, Ctx};
                _                       -> {?AUTH_HEAD, RD, Ctx}
            end;
        _ -> {?AUTH_HEAD, RD, Ctx}
    end.


%% ETAG support

generate_etag(RD, Result) ->
    {mochihex:to_hex(erlang:phash2(Result)), RD, Result}.


%% DELETE support

delete_resource(RD, Ctx) ->
    os:putenv(wrq:path_info(env, RD), []),
    {true, RD, Ctx}.

Simple Webmachine Extension (3/4): ETags

This is the third post in a four-part series about extending a simple Webmachine resource. The first part discussed adding support for the HTTP method PUT, and the second part added basic authorization.

ETag

If I’m sharing management of a server with someone else, I want to be careful of overwriting that other person’s changes. For instance, I might only want to engage “DANGER_MODE” if I can be sure that “COAST=clear”.

I need to know that between my last GET and my next PUT, that the value of the “COAST” variable hasn’t changed. I can handle this simply by generating an ETag for the resource:

-export([generate_etag/2]).

generate_etag(RD, Result) ->
    {mochihex:to_hex(erlang:phash2(Result)), RD, Result}.

Now I can issue conditional requests. When I GET /_env the response will have an ETag header, which is reasonably guaranteed to change if the environment variables change. I can take that ETag and toss it back in an If-Match, and the request will only succeed if the environment variables are in the same state as I last saw them (because the ETag won’t match otherwise). That is:

$ curl -u webmachine:rules -X PUT -H "If-Match: LAST_ETAG" \
   -H "Content-type: application/json" http://localhost:8000/_env/ \
   -d "{\"DANGER_MODE\":\"engaged\"}"

will only succeed if the enviroment is in the state it was when I issued the request that returned Etag: LAST_ETAG (when, hopefully, I checked to make sure that the coast was clear).

Update: part four is up.

Simple Webmachine Extension (2/4): Authorization

This is the second post in a four-part series about extending a simple Webmachine resource. The first part discussed adding support for the HTTP method PUT.

Authorization

Something about modification of server state screams, “Password protection!” at me. Let’s guard the PUT method with Basic auth:

-export([is_authorized/2]).

-define(AUTH_HEAD, "Basic realm=MyOSEnv").

is_authorized(RD, Ctx) ->
    case wrq:method(RD) of
        'PUT' -> basic_auth(RD, Ctx);
        _     -> {true, RD, Ctx}
    end.

basic_auth(RD, Ctx) ->
    case wrq:get_req_header("Authorization", RD) of
        "Basic "++Base64 ->
            case string:tokens(base64:mime_decode_to_string(Base64), ":") of
                ["webmachine", "rules"] -> {true, RD, Ctx};
                _                       -> {?AUTH_HEAD, RD, Ctx}
            end;
        _ -> {?AUTH_HEAD, RD, Ctx}
    end.

Arbitrary decisions:

  • Only PUT is protected. If GET and HEAD should be protected as well, just replace the body of is_authorized/2 with the body of basic_auth/2

I need to update my curl command if I don’t want to be told I’m unauthorized:

$ curl -u webmachine:rules -X PUT -H "Content-type: application/json" \ 
   http://localhost:8000/_env/MY_VAR -d "\"yay\""

Come back tomorrow for part three, where I add a modicum of atomicity.

Update: part three is up.

Follow

Get every new post delivered to your Inbox.