Archive for the ‘Development’ Category
In case you’re using this blog’s feed to get updates about new features on BeerRiot, here’s a notice that BeerRiot now has its own feeds. :)
I’m pretty green to the RSS scene, though, so if I’ve committed some terrible crime against RSS, please let me know, and I’ll get on fixing it. Or, if there’s just some way that I could improve the experience, which all of you RSS-fiends know about, those suggestions are most welcome.
I know what some of you are thinking. “Forums?! Every site under the sun has forums. What did you waste weeks on forums for?” Clearly, if there was an ounce of sense in my head, I would have grabbed the nearest PhpBB implementation, tossed it up, and been done with it.
Well, I agree.
Except that I just couldn’t bear having such a disconnected, tacked on hunk of lameness. Even assuming that people wouldn’t mind the pain in the neck of having a second logon for a forum system, what I really wanted was a system that could really relate to the rest of the content on the site.
For example, I expect people to want to talk about a group (or several) of beers. Maybe someone will have a question about how two styles relate (porter and stout, hefeweizen and wheat, …), or they will have just returned from an event with an interesting selection. I think they should be able to easily set up a discussion about exactly the set of beers they want. So, since we were already doing this to some extent with tags, I have made it a key feature of discussion topics that they should be able to have tags associated with them.
I also expect people to want to get the word out about upcoming beer events. They’ll want to tell people where, and when, but then also discuss how it went afterward. So, in the events forum, all topics are allowed to have dates and locations associated with them.
As such, I went on my merry way developing. An opportunity to sling more Erlang code – how could I resist? ;)
As if that weren’t bad enough, there was the realization late in the project that the new forum system was so much better than the beer comment system that there was no way I could leave the elder in place. So, out came the old, and in went the new. And hey – I always wanted to let people comment directly on breweries, why not add that in too?
If you’ve been anywhere near the software industry (or, in fact, I suspect any industry), you recognize now the project spiral. “Just one more thing, then it will be ready.” That’s how it went for the first couple of weeks. Luckily I recognized the spiral at that point too, and cut off some of the broader plans to get this pushed out (don’t worry – they’re still on the agenda, just on a different deadline).
Believe it or not, I believe I was lucky enough to be able to pick joints at which to truncate plans, that after this big, bulky update I should be able to roll out some nice, smaller updates fairly quickly and smoothly. The next news item shouldn’t be three weeks away. ;)
So that’s that. In case you’re wondering, no, I still don’t regret choosing Erlang/ErlyWeb for this project. I didn’t run up against anything in this development that this system was incapable of handling. As mentioned above, I likely would have been delving deep into the guts of any forum system I chose, so development time would have been just as long. Getting to do it in a language that doesn’t annoy me [yet] was a real treat.
…I’m still alive and plugging away at BeerRiot code. I thought you all might just like to hear that despite the fact that I haven’t made a post here in a while, or updated the news section on the site, development is far from dead. There are things in the works, and I hope they will be ready soon.
But, I’m running into the oldest code-project slow-down in the book. New code is making me rethink old code. Especially since I was much more of an Erlang amateur at the start of this project, my old code often looks ugly now. So, since my new code is doing very similar things to some of the oldest code here, I’m reworking the old stuff – mainly to avoid duplication, of course, but also because I understand the domain better now.
Unfortunately for you, the user, I’m still having quite a fun time slinging Erlang around, so it’s hard for me to just say, “Enough – post it!” But, I think I’ve defined the final goals, and the web of changes is starting to contract. The updates should happen soon (probably within a week), and then there will be lots of new toys to play with. :)
 Okay, so actually, there are older slowdowns in the book, but I ran into them too (that is, real life has demanded extra attention recently). But I think there’s a lull in them for a while, so things should progess faster now.
Just a quick note this evening: BeerRiot was down from about 3:30am to 8am this morning. The MySQL server shut down. I restarted it as soon as I found it down.
Interesting: The MySQL server shut down, but the ErlyDB connection to it was still alive and waiting. As soon as the MySQL server came back up, ErlyDB automatically reconnected, and the site was live.
Sounds like as good of a reason as any to consider switching to Mnesia. I have to assume it’s easier to monitor that than to watch/restart MySQL.
I bet there are a lot of people still questioning the utility of hot code loading. Especially in the web app field, it can seem a little gratuitous. PHP apps don’t need any special hot load facility – the script just gets reread from disk every once in a while.
Well, even if we ignore that there are likely parts of web apps that do need to run all the time, and are not just executed at request time, there’s still the web server to think about. And, guess what I did last week.
Yaws provides lots of nice utility functions. One in particular is yaws_api:htmlize/1, which takes an IoList as an argument, and returns the “same” list with the four big offenders (ampersand, double-quote, less-than, and greater-than) replaced with their HTML entities (& et al.). This function does exactly what you need when serving HTML directly to a modern web browser.
Unfortuntately, htmlize/1 doesn’t work perfectly when sending “FBML” to Facebook. During Facebook’s translation, it converts all characters with ASCII values greater than 127 to unrecognizeable characters, which come out as some form of “?” in a browser.
The fix is simple – just HTML-encode all characters over 127 as HTML entities of the form &#X;, where X is the decimal representation of the ASCII value. In Yaws 1.68, just add the following two lines just before yaws_api.erl:590 (the line with the guard for integer(X)):
htmlize_l([X|Tail], Acc) when integer(X), X > 127 -> htmlize_l(Tail, [$;, integer_to_list(X), $#, $&|Acc]);
Compile the new code by running make in the base directory of the Yaws source. If you're running Yaws from a directory other than the source directory, copy ebin/yaws_api.beam to that other ebin directory.
Edit: See dbt's comment for a way to skip the next paragraph in the simple case.
Now for the magic. My prefered way to load new code is to open a console to the web server's erlang node. First, run "erl -sname Name", where Name is any name other than that of your webserver. Once erl starts up, type C-g (control-g, for you non-Emacs-ers). You'll be asked for a "User switch command". Typing "h" will get you help here, but what you actually want to do is type "r yaws@host", where "yaws@host" is the node name of the webserver's erlang node. Typing "j" should now show two shell sessions. Connect to the second one with "c 2".
Now that you're connected to your web server's erlang node, just type "l(yaws_api)" to load the new code. Any module calling into the yaws_api module will now automatically use the new code. Meanwhile, any code that was in the middle of a yaws_api module call will finish the call with the old code.
Edit: You won't need the next paragraph either, if you followed dbt's instructions.
When you're done mucking about (I know you've just spent the last half hour figuring out what other bits of your webserver you can touch from here), type C-g again, then kill the remote console with "k 2". Connect back to your first console with "c 1", then exit it in the normal manner.
So, voila, Á now comes out of htmlize/1 as Á. International beer names show up properly in Facebook, and (oh boy!) BeerRiot Local now works in IE6 (which couldn't parse those letters from the XML tag file). Two birds, one stone, I love it.
On a couple of side notes:
Thanks for the versioning system suggestions. I've settled on Mercurial for now, and I'm quite happy with it so far. Bit of a pain upgrading Python versions, but I probably should have done that long ago anyway.
And, I've been doing more than just building a website around beer this summer. I've also been growing my own ingredients! I made my first hop harvest earlier this week. It was only an ounce wet, which turned into about 1/8 oz. dry, but I was proud to have some success anyway. Here's the proof:
As promised, here’s the post about the development process of the BeerRiot Facebook app.
Aside: Wow, rereading that last post about the Walled Gardens, I can’t believe I posted it. Totally fluffy lameness. Forgive me – this post will not be as bad.
I could very easily make this another post like others out there right now. Many people are upset about how unfinished the Facebook API is. Documentation is lacking (and only available online), specifications are weak, and test environments could be more feature-rich.
But, except for that little digression above, I promise, this is not one of those posts.
Instead, I’d really like to remark about how great it is developing this app in Erlang and Erlyweb!
There are nearly a dozen different Facebook API implementations – every object-oriented web language you can name. Facebook’s own official one is PHP5. I could have used the PHP version – Yaws comes complete with the ability to run PHP scripts. But, since the rest of BeerRiot is already in Erlyweb…
I rewrote the Facebook client in Erlang. And truthfully, it really wasn’t that bad at all. Erlang comes with very simple HTTP-communciation and XML-munging code. Hell, there’s even a simple way to compute an md5 sum. Once I figured out how to actually put together a proper POST and walk a rather verbose XML tree, the rest was just mimicking as closely as possible the official PHP scripts.
All this is not to say the process wasn’t without difficulties. Yaws’ standard url_encode proceedure doesn’t like nested io_lists very much, so I’m doing some ugly string/binary flattening. And, Facebook’s errors are nearly meaningless … but this isn’t about Facebook complaints. ;)
There are yet more benefits. Rather than dealing with cron firing up processes periodically, I have fully-supervised Erlang gen_servers up to date constantly with what data needs to be pushed out to Facebook. I can log into the running server and examine the current active sessions, monitor profile data pushes, etc.
Extending my existing Erlyweb application to handle new Facebook requests from the canvas page was even pretty simple (once I finally – I think – understood the Facebook session rules). Another controller, another view – bam!
One of my few real pains was source control. I may need to finally bite the bullet and leave CVS. I’m hearing good things about SVN. But that’s another topic.
So, chalk up one more win for Erlang/Erlyweb. Fantastic existing libraries. Quick development environment. Crazy server flexibility. I’m staying as long as I can.
Yes, I promised new things well over a week ago. My excuse is that I picked up a cold during my trip to the Middle of Nowhere that took me out of commission for a solid week. :P But, now that that is over…
I’ll probably talk more about how the development went in another post. Right now, though, I’m thinking over the whole “Walled Garden” argument.
For many of the apps, both the data source and the data sink reside inside the Garden – Facebookers talk to other Facebookers. For another large chunk, the data source is outside, but the sink is still inside – Stock quotes, sports scores, etc.
For BeerRiot, I’ve managed to allow the data source as well as the data sink to be on either side of the wall. Beers and comments added by Facebookers can be seen by non-Facebookers, and the other way around. Really, I’m just using Facebook something like an OpenID provider.
Of course, for some apps this isn’t necessary, and for others it may be impossible. But, it feels like this is the right way to think going forward. Using this method, there would be nothing stopping me from bridging more walls to connect more gardens together, while still keeping the web at large in the loop.
Okay, I’m about to fall asleep, and I can’t tell if this blog post is just simple fluff or not, but I’m going to post it anyway. Post in the comments if you like/dislike the FB app, what you think of Walled Gardens, or how bad this post really is.
I’m not a fan of “settings” pages. They give a terrible interaction experience.
To change settings on a settings page, I have to stop what I’m doing; open the settings page; find, change, and often save the setting; then find my way back to what I was doing. If my change wasn’t exactly what I wanted, I have to do it all over again.
More often than not, this breaks my concentration to the point where it may take me minutes to figure out what I had been doing, and get back to it. Productivity tends toward zero.
So, I’m trying something different with BeerRiot. I’m going to try to keep all settings relevant to the page they affect on the page they affect. No need to leave to find the setting elsewhere – just change it and update.
I forsee only a couple of occasions that I’d ever have to make an exception: the structures to change the setting would consume too much space, or the setting doesn’t relate to any other page. The latter is already exemplified by email and password settings.
I think these exceptions being on separate pages may make sense, though. If the structures take a lot of space, they may be complicated, so focusing on them alone may be helpful. If they’re not related to the page I’m on, then I’m probably not worried about getting back to that page after I change them.
Today I have the default map view setting on the map view page, maximum item list lengths on each item list page, and each member’s public information on their own public page. I think they fit pretty well, but of course I’d like to hear what you all think.
Also, does anyone know of any research on this topic? I’ve read a bit of general user-interface stuff, but I don’t recall anything that specifically spoke to “settings” pages.
Don’t Throw Out Maybe Yet!
Jeffrey Zeldman’s post “Maybe” is one option too many caught my eye this afternoon. I completely agree with him that “maybe” is a pain in the neck. From my vantage point, yes/no (or like/dislike) are the only user inputs that really matter.
Most of you will notice that BeerRiot has a third option – “shrug”. It wasn’t always this way. The first versions of BeerRiot had no middle ground – you liked a beer, or you disliked it. I was staunch in my position, callously telling people that they should make up their mind.
It wasn’t until over half of the first twenty people to which I showed BeerRiot asked, “Where’s ‘meh’?” that I started to change my thinking. I started to realize that being forced to make a black/white decision was making people uneasy. They really wanted the comfort of being able to postpone making their choice.
The decision was pretty easy at that point. I wanted people to stay around my site. Therefore, it seemed a good idea to make them comfortable. Shrug was born.
In the time since introducing shrug, I’ve found it to be more than just indecision. It’s a way to acknowledge the existence of a thing, without passing judgement. For beer voting, it lets me know that although I have no strong feelings about a beer, I have tried it (something I have trouble remembering with the wide variety available). I can see it working similarly for party invites: it lets the organizer know that even though I don’t know my schedule, I have, in fact, received the invitation.
Here’s another way to think about it. Suppose there is no “maybe” option. You must either accept or decline the invitation. So, what are people who don’t respond? Are they not “maybe”? Maybe you’re a pessimistic person: you consider non-responders “no”. Okay, then, if all non-responders are “no”, then why even have a “no” option? Why not just have “yes”? A unary system – respond “yes”, or you don’t exist.
For BeerRiot, at least, that would seriously cut my dataset. I think the same would be true for party invites. You couldn’t even tell what kind of buffer you might want to plan for – all you would know is the number of people who are definitely coming. Or in BeerRiot’s case, I couldn’t tell if a beer is controversial – I’d only know the number of people who like it.
So, I say don’t throw maybe out yet. It’s more than just indecision. It’s user comfort and metadata all rolled into one.
I’ll leave you with this, though: I don’t see the point in more than three choices. “Kind-of-like” and “Sort-of-dislike” are even less data than maybe to me. At least with “maybe” I can tell the voter is unsure. With “maybe-like”, I can’t tell if she’s unsure or genuinely less positive about this beer than others.
P.S. I love Bill W.’s comment about rating other people’s ratings. It’s exactly the kind of problem that BeerRiot tries to solve.