Archive for April, 2009|Monthly archive page

Simple Webmachine Extension (4/4): DELETE

This is the third post in a four-part series about extending a simple Webmachine resource. The first part discussed adding support for the HTTP method PUT, and the second part added basic authorization, while the third part added conditional requests through ETag.

DELETE

I can modify variables through PUT; how about removing them altogether with DELETE?

First a little sidetrack. While Erlang exposes os:getenv/1 and os:putenv/2, it does not expose an os:deleteenv/1. So, what I’m going to do instead is adopt the convention that a variable set to “nothing” (i.e. the empty list) has been deleted. This requires a little modification to resource_exists/2 to filter out variables with empty-list values. (See the full source at the end of this post.)

Sidetrack #2: I also noticed that there was a bug preventing values from containing the equals sign, so while hacking resource_exists/2, I fixed that as well.

With the empty-list-is-deleted convention in hand, I can implement DELETE handling like this:

-export([delete_resource/2]).

delete_resource(RD, Ctx) ->
    os:putenv(wrq:path_info(env, RD), []),
    {true, RD, Ctx}.

I only want to allow one variable to be deleted at a time, so I’ll modify allowed_methods/2 such that it only adds 'DELETE' to the method list if the request path is of the form /_env/VARIABLE.

I also want DELETE to require authorization, so I’ll modify is_authorized/2 to watch for both PUT and DELETE.

And that’s that. I can now use the following curl command to delete MY_VAR:

$ curl -u webmachine:rules -X DELETE http://localhost:8000/_env/MY_VAR

Wrap Up

Before I get to the complete source, I want to take a moment to highlight something I stressed in an earlier blog post: Did you notice that none of the code I’ve displayed in the last few days has mentioned specific HTTP response codes?

The env_resource now runs all over the HTTP decision flowchart, returning everything from 405 when methods other than GET, HEAD, PUT, and DELETE are issued (or just the first three in the whole-environment case), to 401 when PUT or DELETE is attempted without the proper credentials, and 412 when ETags don’t match on a PUT (or 304 when they do match on a GET!), and yet, I didn’t mention a single code. I simply described the properties I wanted the resource to have, and let Webmachine do the translation.

I stress this because I think it’s important to consider the power of an HTTP translation system. I’ve seen reduced development time as an effect of not having to worry about getting to the right response code in every corner case, while still adding necessary headers. I’ve also seen reduced troubleshooting time as an effect of being able to read through a resource that only concerns itself with describing its properties, rather than including a lot of mechanics for tearing apart HTTP requests and building up HTTP responses.

Plug

If this series has raised your interest level in Webmachine, I recommend you attend Justin Sheehy’s talk at the Bay Area Erlang Factory.

The Complete Code

%% dispatch:
%% {["_env"],      env_resource, []}.
%% {["_env", env], env_resource, []}.

-module(env_resource).
-export([init/1, content_types_provided/2, resource_exists/2, to_json/2]).
-export([allowed_methods/2, content_types_accepted/2, from_json/2]).
-export([is_authorized/2]).
-export([generate_etag/2]).
-export([delete_resource/2]).
-include_lib("webmachine/include/webmachine.hrl").

init(_) -> {ok, undefined}.

content_types_provided(RD, Ctx) ->
    {[{"application/json", to_json}], RD, Ctx}.

resource_exists(RD, Ctx) ->
    case wrq:path_info(env, RD) of
        undefined ->
            Result = [ {K, string:join(V, "=")}
                       || [K|V] <- [ string:tokens(E, "=")
                                     || E <- os:getenv() ],
                          V /= [] ],
            {true, RD, {struct, Result}};
        Env ->
            case os:getenv(Env) of
                false  -> {false, RD, Ctx};
                []     -> {false, RD, Ctx};
                Result -> {true, RD, Result}
            end
    end.

to_json(RD, Result) ->
    {mochijson:encode(Result), RD, Result}.


%% PUT support

allowed_methods(RD, Ctx) ->
    {['GET', 'HEAD', 'PUT'
      |case wrq:path_info(env, RD) of
          undefined -> [];
          _         -> ['DELETE']
       end],
     RD, Ctx}.

content_types_accepted(RD, Ctx) ->
    {[{"application/json", from_json}], RD, Ctx}.

from_json(RD, Ctx) ->
    case wrq:path_info(env, RD) of
        undefined ->
            {struct, MJ} = mochijson:decode(wrq:req_body(RD)),
            [ os:putenv(K, V) || {K, V} <- MJ ];
        Env ->
            MJ = mochijson:decode(wrq:req_body(RD)),
            os:putenv(Env, MJ)
    end,
    {true, RD, Ctx}.


%% AUTH support

-define(AUTH_HEAD, "Basic realm=MyOSEnv").

is_authorized(RD, Ctx) ->
    case wrq:method(RD) of
        PD when PD == 'PUT'; PD == 'DELETE' -> basic_auth(RD, Ctx);
        _                                   -> {true, RD, Ctx}
    end.

basic_auth(RD, Ctx) ->
    case wrq:get_req_header("Authorization", RD) of
        "Basic "++Base64 ->
            case string:tokens(base64:mime_decode_to_string(Base64), ":") of
                ["webmachine", "rules"] -> {true, RD, Ctx};
                _                       -> {?AUTH_HEAD, RD, Ctx}
            end;
        _ -> {?AUTH_HEAD, RD, Ctx}
    end.


%% ETAG support

generate_etag(RD, Result) ->
    {mochihex:to_hex(erlang:phash2(Result)), RD, Result}.


%% DELETE support

delete_resource(RD, Ctx) ->
    os:putenv(wrq:path_info(env, RD), []),
    {true, RD, Ctx}.

Simple Webmachine Extension (3/4): ETags

This is the third post in a four-part series about extending a simple Webmachine resource. The first part discussed adding support for the HTTP method PUT, and the second part added basic authorization.

ETag

If I’m sharing management of a server with someone else, I want to be careful of overwriting that other person’s changes. For instance, I might only want to engage “DANGER_MODE” if I can be sure that “COAST=clear”.

I need to know that between my last GET and my next PUT, that the value of the “COAST” variable hasn’t changed. I can handle this simply by generating an ETag for the resource:

-export([generate_etag/2]).

generate_etag(RD, Result) ->
    {mochihex:to_hex(erlang:phash2(Result)), RD, Result}.

Now I can issue conditional requests. When I GET /_env the response will have an ETag header, which is reasonably guaranteed to change if the environment variables change. I can take that ETag and toss it back in an If-Match, and the request will only succeed if the environment variables are in the same state as I last saw them (because the ETag won’t match otherwise). That is:

$ curl -u webmachine:rules -X PUT -H "If-Match: LAST_ETAG" \
   -H "Content-type: application/json" http://localhost:8000/_env/ \
   -d "{\"DANGER_MODE\":\"engaged\"}"

will only succeed if the enviroment is in the state it was when I issued the request that returned Etag: LAST_ETAG (when, hopefully, I checked to make sure that the coast was clear).

Update: part four is up.

Simple Webmachine Extension (2/4): Authorization

This is the second post in a four-part series about extending a simple Webmachine resource. The first part discussed adding support for the HTTP method PUT.

Authorization

Something about modification of server state screams, “Password protection!” at me. Let’s guard the PUT method with Basic auth:

-export([is_authorized/2]).

-define(AUTH_HEAD, "Basic realm=MyOSEnv").

is_authorized(RD, Ctx) ->
    case wrq:method(RD) of
        'PUT' -> basic_auth(RD, Ctx);
        _     -> {true, RD, Ctx}
    end.

basic_auth(RD, Ctx) ->
    case wrq:get_req_header("Authorization", RD) of
        "Basic "++Base64 ->
            case string:tokens(base64:mime_decode_to_string(Base64), ":") of
                ["webmachine", "rules"] -> {true, RD, Ctx};
                _                       -> {?AUTH_HEAD, RD, Ctx}
            end;
        _ -> {?AUTH_HEAD, RD, Ctx}
    end.

Arbitrary decisions:

  • Only PUT is protected. If GET and HEAD should be protected as well, just replace the body of is_authorized/2 with the body of basic_auth/2

I need to update my curl command if I don’t want to be told I’m unauthorized:

$ curl -u webmachine:rules -X PUT -H "Content-type: application/json" \ 
   http://localhost:8000/_env/MY_VAR -d "\"yay\""

Come back tomorrow for part three, where I add a modicum of atomicity.

Update: part three is up.

Simple Webmachine Extension (1/4): PUT

I was in need of a break last night after flailing in the face of a new library-language-build-system for a few hours. So, I decided to hack some Webmachine (it was too late to sit down at the trap set).

I was thinking about how the os-environment resource from my last post could be extended. This post begins a four-part series in which new capabilities are added to env_resource.erl.

PUT

Let’s start with modification. Why not allow setting those variables?

I need to do three things: announce that the resource supports PUT, list what types of data it accepts, and then actually handle decoding the incoming data. Fifteen quick lines of code should handle it:

-export([allowed_methods/2, content_types_accepted/2, from_json/2]).

allowed_methods(RD, Ctx) ->
    {['GET', 'HEAD', 'PUT'], RD, Ctx}.

content_types_accepted(RD, Ctx) ->
    {[{"application/json", from_json}], RD, Ctx}.

from_json(RD, Ctx) ->
    case wrq:path_info(env, RD) of
        undefined ->
            {struct, MJ} = mochijson:decode(wrq:req_body(RD)),
            [ os:putenv(K, V) || {K, V} <- MJ ];
        Env ->
            MJ = mochijson:decode(wrq:req_body(RD)),
            os:putenv(Env, MJ)
    end,
    {true, RD, Ctx}.

Arbitrary decisions were made above:

  • Clients shall send JSON-encoded data. I could have just as easily added another element to the return value of content_types_accepted/2 and written another method to handle it (e.g. {"text/plain", from_text} and from_text/2). Accepting JSON is nice, since it makes PUT and GET simply symmetric.
  • /_env expects a JSON structure, while /_env/VARIABLE expects a string. Again with the simple symmetry between PUT and GET.
  • /_env only modifies or creates the variables described in the JSON structure it receives. Alternatively, it could have cleared out any unnamed environment variables, but this seemed unnecessary.
  • No body is returned in a successful response. It would have been fairly simple to generate the same body that would have been returned in a GET, then use wrq:append_to_response_body/2 to return a modified RD, but this also seemed unnecessary.

I can now set MY_VAR to "hello", using two different curl commands:

$ curl -X PUT -H "Content-type: application/json" \
   http://localhost:8000/_env -d "{\"MY_VAR\":\"hello\"}"

$ curl -X PUT -H "Content-type: application/json" \
   http://localhost:8000/_env/MY_VAR -d "\"hello\""

Come back tomorrow for part two, in which I’ll add authorization via username and password.
Update: part two is up.

Simple Webmachine – Proper HTTP Resources

Update: This post sparked my Webmachine nerve in such a way that I wrote four more posts about extending the resource described below. Read them if you’d like to see how this resource can evolve.

This morning I read a post about CouchDB’s HTTP Handlers. Jón Grétar Borgþórsson demonstrates how one might implement a handler that serves a JSON structure describing the OS environment variables.

I thought that two additional pieces of code might be interesting. I’ll lead with an example showing the most likely way this resource would have been coded for Webmachine:

%% dispatch:
%% {["_env"],      env_resource, []}.
%% {["_env", env], env_resource, []}.

-module(env_resource).
-export([init/1, content_types_provided/2, resource_exists/2, to_json/2]).
-include_lib("webmachine/include/webmachine.hrl").

init(_) -> {ok, undefined}.

content_types_provided(RD, Ctx) ->
    {[{"application/json", to_json}], RD, Ctx}.

resource_exists(RD, Ctx) ->
    case wrq:path_info(env, RD) of
        undefined ->
            Result = [ list_to_tuple(string:tokens(E, "="))
                       || E <- os:getenv() ],
            {true, RD, {struct, Result}};
        Env ->
            case os:getenv(Env) of
                false  -> {false, RD, Ctx};
                Result -> {true, RD, Result}
            end
    end.

to_json(RD, Result) ->
    {mochijson:encode(Result), RD, Result}.

The biggest difference to note between the Webmachine version and the CouchDB version is that the proper HTTP status codes are returned from the Webmachine resource. The CouchDB handler returns 500 or 405 when the requested resource is not found. The proper status for this case is 404. Webmachine knows this, and handles the choice automatically.

A little extra emphasis, I think, is appropriate here: Webmachine chooses the proper response code for you. You define methods that describe the state of your resource (like whether or not it exists, what methods it allows, etc.), and Webmachine negotiates the muck of HTTP.

As a bonus, the Webmachine resource is the same length, while at the same time being less dense and more readable.

Let’s not get hasty, though. If there is a really good reason for returning an alternate status code, Webmachine won’t get in your way. To prove it, here’s a Webmachine resource that (as near as I can tell, I’m not a CouchDB guru) returns exactly the same statuses as the CouchDB handler:

%% dispatch:
%% {["_env2", '*'], env_resource, []}.

-module(env2_resource).
-export([init/1, content_types_provided/2, resource_exists/2, to_json/2]).
-include_lib("webmachine/include/webmachine.hrl").

init(_) -> {ok, undefined}.

content_types_provided(RD, Ctx) ->
    {[{"application/json", to_json}], RD, Ctx}.

resource_exists(RD, Ctx) ->
    case string:tokens(wrq:disp_path(RD), "/") of
        [] ->
            Result = [ list_to_tuple(string:tokens(E, "="))
                       || E <- os:getenv() ],
            {true, RD, {struct, Result}};
        [Env] ->
            case os:getenv(Env) of
                false ->
                    {{halt, 500},
                     wrq:set_resp_header(
                       "Content-type", "application/json",
                       wrq:append_to_response_body(
                         mochijson:encode(
                           {struct, [{error, "not_found"},
                                     {reason, "Variable Not Found"}]}),
                         RD)),
                     Ctx};
                Result ->
                    {true, RD, Result}
            end;
        _ ->
            {{halt, 405},
             wrq:set_resp_header("Allow", "GET,HEAD", RD),
             Ctx}
    end.

to_json(RD, Result) ->
    {mochijson:encode(Result), RD, Result}.
Follow

Get every new post delivered to your Inbox.