Phil Toland

I will be presenting at the Lone Star Ruby Conference this year with my coworker Mike (a.k.a. @lifeinzembla on Twitter). The talk is titled “Polyglot Parallelism: A Case Study in Using Erlang and Ruby at Rackspace” and will cover our experience rebuilding some older systems in Erlang. We plan to get into when you should use Erlang to augment your existing code as well as what benefits you can expect from Erlang and what pitfalls to avoid. The full abstract is below:

Two years ago Rackspace had a problem: how do we backup 20K network devices, in 8 datacenters, across 3 continents, with less than a 1% failure rate — every single day? Many solutions were tried and found wanting: a pure Perl solution, a vendor solution and then one in Ruby, none worked well enough. They were not fast enough or they were not reliable enough, or they were not transparent enough when things went wrong.

Now we all love Ruby but good Rubyists know that it is not always the best tool for the job. After re-examining the problem we decided to rewrite the application in a mixture of Erlang and Ruby. By exploiting the strengths of both — Erlang’s astonishing support for parallelism and Ruby’s strengths in web development — the problem was solved.

In this talk we’ll get down and dirty with the details: the problems we faced and how we solved them. We’ll cover the application architecture, how Ruby and Erlang work together, and the Erlang approach to asynchronous operations (hint: it does not involve callbacks). So come on by and find out how you can get these two great languages to work together.

Phil Toland does not have an impressive beard, but he has been working with Erlang since 2008 when he cut his Erlang teeth developing Yet Another Dynamo Clone. These days he works at Rackspace automating network devices against their will and often his own better judgement.

Mike Sassak is Internet famous for his observational skills and his impressive beard. Also, he works at Rackspace and contributes to Cucumber.

I needed to encrypt some text before storing it in the database. Since I needed to get the original back at some point I settled on Blowfish and AES as good candidates. There are a couple gems with provide this type of encryption, but I could not get either one to work.

Eventually I came across this snippet and this blog post. It turns out that it is trivial to use the Ruby OpenSSL library to do encryption. I cleaned up the code from the first snippet a bit:

require 'openssl'

module Blowfish
   def self.cipher(mode, key, data)
     cipher = OpenSSL::Cipher::Cipher.new('bf-cbc').send(mode)
     cipher.key = Digest::SHA256.digest(key)
     cipher.update(data) << cipher.final
   end

   def self.encrypt(key, data)
     cipher(:encrypt, key, data)
   end

   def self.decrypt(key, text)
     cipher(:decrypt, key, text)
   end
 end

 if $0 == __FILE__
   p "text" == Blowfish.decrypt("key", Blowfish.encrypt("key", "text"))
 end

The Obama administration is announcing steep increases in the federal CAFE standards. Under the new standards average fuel efficiency will increase from 27.5 to 35.5 MPG. The administration is positioning it as a win for everybody: the government continues to assert its control over the auto industry, environmentalists get thrown a bone and the auto manufacturers get big “green” subsidies. It is good for everyone but consumers, anyway. The auto manufacturers will of course pass the cost of these changes along to consumers to the tune of $1000 per car. John Stossel makes an insightful observation:

In a press release, the administration admits that the changes will increase the cost of the typical car by $1,000. But they say it will save each driver $3,000 over the life of the car. Are they right? I don’t know, but if these savings really exist, why does the government need to FORCE people to take them?

If automakers wanted to make cars like this, they would do it on their own.

The problem is that Americans generally don’t want fuel efficient vehicles. They want SUVs, minivans, luxury and sports cars. The CAFE standards contributed to the downfall of the Big 3 automakers last year as they churned out crappy fuel efficient econoboxes that nobody wanted so that they could continue selling the SUVs people did want. I don’t see that changing significantly unless gas gets way more expensive.

ISO OOXML convener: Microsoft’s format “heading for failure”:

Although Microsoft’s Office Open XML (OOXML) document format became an ISO standard two years ago, the company still hasn’t built any software that truly complies with the standard. Microsoft Office 2010, which is expected to be released later this year, implements the deprecated “transitional” version of the format and is not compatible with the version that was crafted by the standards body.

Everyone who is surprised by this please raise your hand. Yeah, that is what I thought. I was skeptical of the ISO OOXML standardization process from the beginning. Microsoft has never been big on respecting standards, and it appears that is true even when they create the standard themselves.

I have release version 0.4.6 of Patron, my HTTP client library for Ruby. The big news for this release is a dramatic speedup for uploads. There are still a few bugs outstanding and I hope to have a new release ready soon. You can get Patron via rubygems with gem install patron or get the source from Github.

When I decided to move my blog from Wordpress to Tumblr I did a lot of searching to find a tool that would import my content. I found a couple of solutions, but none that worked for me. After taking a look at Tumblr’s API docs I decided to throw together a simple script that would parse a Wordpress export file and post the content to Tumblr. It turned out to be pretty easy. Since this is one of those scratch an itch type of solutions it is biased a bit towards the type of content I needed to import (mostly plain posts formatted with Markdown). I am posting it in the hopes that it will be useful, but as always, caveat programmor.

This is going to be one of those embarrassingly meta posts where I talk about the technology behind this site. Rather than drag this out needlessly I will get right to the point and say that I have fully moved my blog over from a self-hosted Wordpress install to Tumblr. Actually, it is a little more complicated than that. I had three sites where I theoretically posted content: my main blog at fiatdev.com, my personal site at philtoland.com and my Tumblr blog at tumbl.fiatdev.com. While I have not posted regularly for a couple of years, it became clear to me that I simply had too many buckets for online content. It was time to simplify and so I moved the content from all three sites into my Tumblr blog.

Quite frankly, I am tired of having to maintain a blog engine. That is something that was kinda fun when I started but has become a chore for me. I began to obsess over the Wordpress engine itself while not writing any actual content. I like Tumblr because it is hosted and simple. Oh, and the bookmarklet is “teh awesome” as the kids say these days. Tumblr’s bookmarklet and simplicity remove the friction from posting content (hence the title of this post).

Now, with all of that having been said, we will see if any of this results in more content being posted here or not.

I recently reset my iPhone in an attempt to clear up some weird issues I was having. It wasn’t until afterwards that I realized that I had forgotten to pull my photos off of the phone first. After doing some searching I discovered that the iPhone backup files are located at:

/Users/<user>/Library/Application Support/MobileSync/Backup

I then found the bkupextract.pl script that attempts to extract all of the sqlite3 databases from the backup files. I modified the script to also extract the plist, JPEG and PNG files and posted it to a gist. My Perl-Fu is weak, so no complaining about the quality of the code, but it found my photos.

The 0.4 version of Patron has been released. This version addresses feedback from the initial public release and fixes a few minor bugs:

• Fixed a bug in the Patron.version method
• Added HTTP proxy support
• Enabled SSL and Win32 support
• Added ability to download data to a file with GET
• Fixed a bug where header values were prefixed with a space
• Made the Session#request method public
• Added support for the WebDAV COPY method
• Added support for uploading files with POST and PUT

In addition to all of this, Patron has a new home page at http://toland.github.com/patron/.

So, what’s next for Patron? Well, I would like to put out a 0.5 release in the next few weeks with some refactoring and API tuning. Also, I would like to implement the rest of the WebDAV methods. I am toying with the idea of creating a net/http compatible interface so that Patron can be used as a drop-in replacement. Not sure if that is going to go anywhere, though.

Once I am happy with the API and documentation I will release 1.0. Other than that, I try to respond to bug reports and feature requests in a timely manner. I originally wrote Patron to scratch my own itch, and that itch has been scratched. At this point I am looking to make Patron as broadly useful as I can, and your feedback helps with that.

Patches and contributed documentation are always welcome. In particular, if anyone feels like trying to get this working on Windows, I will be happy to integrate the resulting patches and publish a binary gem.

One thing that I did not like about working with the Nitrogen web framework is the way that view rendering is handled. Nitrogen calls the function main/0 in your web modules which, by convention, returns an instance of the template record. This record points to a template file that has HTML code with a little Erlang mixed in. Essentially, you can call functions in your web module and whatever content those functions return is interpolated into your template.

This all makes it sound a little more complicated than it really is. Here is a little “hello world” sample to illustrate how it works. First, the web_page Erlang module defines the main/0 function and a couple of callouts:

-module(web_page).
-include("wf.inc").
-compile(export_all).

main() -> #template { file="./wwwroot/template.html" }.
title() -> "Hello, World!".
content() -> #h1 { text="Hello, World!" }.

This will cause Nitrogen to render the template.html file.

<html>
<head>
<title>[[[page:title()]]]</title>
</head>
<body>
[[[page:content()]]]
</body>
</html>

The callouts in the template will call the respective functions in the web_page module and we end up with a page that displays a bold “Hello, World!”. The template file is really analogous to a Rails layout and the content that Rails puts into a view template is returned from the callouts using a markup DSL based on the Erlang record syntax.

This template mechanism is simple, but it becomes very limiting when working on a non-trivial application. Nitrogen mixes controller functionality with presentation in a way that is reminiscent of PHP or JSP. Also, you cannot compose templates and the markup DSL doesn’t cover all of the HTML elements and attributes.

I ran into this problem while working on a Nitrogen powered site and decided that I just couldn’t live with the default template system. After a little digging around I discovered ErlyDTL which implements a substantial subset of the Django Template Language in Erlang. It turns out that using ErlyDTL in a Nitrogen site is quite simple.

The first thing I needed was a utility function to render ErlyDTL templates. Nitrogen includes implicit elements for the notification flash and Javascript code for all of that AJAX goodnes which I needed to replicate in my render function.

-module(my_util).
-compile(export_all).

template_path(Template) ->
    filename:join([code:priv_dir(my_app), templates, Template]).

render_template(Name, Variables) ->
    Template = template_path(Name ++ ".html"),
    Mod = list_to_atom(Name ++ "_template"),
    ok = erlydtl:compile(Template, Mod),
    {ok, Body} = Mod:render([{flash, element_flash:render()},
                             {script, wf_script:get_script()}
                             | Variables]),
    Body.

Now, I can replace the simple module from earlier with this new one:

-module(web_page).
-include("wf.inc").
-compile(export_all).

main() ->
    my_util:render_template("template", [
          {title, "Hello, World!"},
          {content, content()}]).

content() -> "Hello, World!".       

My template file is now equally simple:

<html>
<head>
<title>{{ title }}</title>
</head>
<body>
<h1>{{ content }}</h1>    
</body>
</html>

All of the markup is now in the template file, and the module is concerned only with getting the content from somewhere and making it available to the view. This also allows us to use the full power of ErlyDTL: we can compose templates and perform simple operations on the content to be displayed. Here is an example from the ErlyDTL home page:

Welcome back, {{ name }}!

You have {{ friends|length }} friends: {{ friends|join:", " }}

Have some primes:
{# this is exciting #}
{% for i in primes %}
    {{ i }}
{% endfor %}

There is one drawback to this approach. The Nitrogen DSL includes helpers for generating links that work with its event system. When using ErlyDTL we have to create those links manually, but it isn’t hard. The key thing to remember is that the link id is the event name that will be passed to your module. Here are a couple of examples:

<a id="do_it" href="javascript:">Do It!</a>
<button id="do_it" class="button submit">Do It!</button>

The extra flexibility of DTL is certainly worth the small amount of extra work.

I was recently given the opportunity to release several of the projects I was working on at The Hive as open source. These projects are libraries that we built to address specific problems we ran into but were not core to our business. Most of these were always destined to be released, it just took a while before we got around to it. Everything is currently on my GitHub page, but I wanted to do a little cleanup on each project and announce them individually.

Today I am announcing the first of these projects: Patron. Patron is an HTTP client library for Ruby that features a usable API and good performance. As for the name Patron, I searched through the thesaurus looking for synonyms for the word “client”.

The performance problems with the built-in Ruby HTTP client are well documented elsewhere, so I won’t go into that here. The other alternative is the Curb gem which is based on libcurl and offers great performance. Unfortunately, curb is unfinished and doesn’t appear to have an official maintainer. Also, the API leaves much to be desired. These issues are compounded by the fact that curb is primarily implemented in C and is difficult to modify.

I tried to patch curb to get all of the features we wanted, but subtle bugs kept creeping in. Eventually I gave in and wrote Patron. The goal was to have a Ruby HTTP client built on libcurl with a reasonable API. By implementing as much as possible in Ruby I think that Patron is much more maintainable than curb.

I have not made an attempt to support every feature of libcurl. Libcurl is huge and trying to support every feature would lead to bloat and make it harder to maintain. Instead, I built a small API that did what we needed it to do. It has been in use at The Hive for some time now and seems pretty stable.

The documentation for Patron is on Rubyforge and the source code is available on GitHub.

I used NetBeans back when it was called Forte for Java and have kept an eye on it ever since. Every few releases I would download it and try it out again. Since Eclipse came onto the scene, NetBeans was just “that other Java IDE.” I no longer work with Java, nor do I use an IDE, but I still download NetBeans every once in a while to see how far it has come. When Rails began to gain traction, the NetBeans team began to work in Ruby support. The Ruby support in NetBeans has always demoed well, but always seemed a bit too clunky for me. And it looked like crap on OS X.

NetBeans 6.7 was released on Monday and I dutifully downloaded it and played with it for a couple of days. The first thing that stuck me is that it looks good on OS X.

This is the best looking Swing application I have ever seen, and yet there are little inconsistencies that can become annoying. For example, you can drag a Cocoa window by clicking anywhere on the integrated toolbar, but in NetBeans you have to click on the title bar. This is a big issue since the toolbar in NetBeans looks like an integrated Cocoa toolbar, but it isn’t. Also, clicking on a file in the explorer on the left shows the file’s members in the Navigator, but does not open the file. You have to double click to actually open a file.

All of this would be forgivable if NetBeans offered better functionality than my current toolset. The whole idea behind IDEs is that it is more convenient to have all of the functionality you need bundled into one application than to use several separate apps. However, getting to some of the functionality in NetBeans requires digging around in the UI, and some stuff is just broken compared to the non-IDE equivalent. For example, when running Ruby tests or the Rails server via script/server from within NetBeans, the ASCII color codes are not properly interpreted making the output unreadable.

So far, I have found NetBeans less convenient than the combination of TextMate, GitX, Terminal and Sequel Pro. Part of this is due to the fact that I have a lot of time invested in my current development stack. The latest version of NetBeans shows enough promise, and I am enough of a tools geek, that I am going to tough it out a few more days. That having been said, I am having a hard time seeing the value in an IDE when the components of a non-integrated development tool stack are of such high quality.