If you read this blog you know I’m a huge fan of Ruby. I have nothing but good things to say about it. I’m dangerously close to being labeled a fanboy. For the longest time I thought I’ll eventually get to redeem myself by linking to an article pointing out some of the downsides of Ruby. Little did I know, I’ll end up writing it.

You may ask yourself, why I’m so pessimistically looking for the downfall of a language I’m invested in? I’m not. I’m looking for gotchas.

Perfect only exists on TV. Every piece of software you work with has warts, gotchas and, unfortunately, sometimes blackholes through which all your energy goes and none comes back. Part of writing software is knowing which pitfalls to avoid, which trapdoors to walk around, and how to navigate around strong centers of gravity. The quicker I build that knowledge, the better tool it becomes.

One thing that always concerned me was meta-programming. Java doesn’t have much of a heritage for meta-programming. Very few brave through thick forests of reflectors, proxies and code enhances, even less come out the other side. Ruby on the other hand, in instrument of precision that can cut through cruft like nobody’s business. But remember what they say, and don’t go running with scissors.

There’s also LISP et al, but these languages always struck me as surgical scissors. Ultimate precision that are painful to use on cardboards, fabrics and the odd electrical cord. I never managed to use those in real life to learn any real lesson.

Try as I may, I never managed to shoot myself in the foot, but I’m usually careful like that. Until today.

method_not_missing

I’m working on a project right not that uses SFTP to upload files onto a remote server. I point the code at the base URL, essentially the root document directory, and let it upload all the files over. The files are set in a structure, so the code creates those remote directories before it can upload the files. In spite of the non-existent documentation, I managed to grok Net::SFTP quickly, part from peeking at the Capistrano source code, and part from being based on Net::SSH.

So now I’m setup and ready to go, I run the first test and it passes successfully. The second run fails, complaining the directories already exist, so I use realpath to check before using mkdir to create. Repeatable runs are successful, as I’m adding progress bar, checksums, cleaning up the code, and testing after every major change. So far so good.

At this point the code is running in a “test” configuration, uploading files to localhost, which like every good Linux box comes pre-installed with SSH/SFTP. When I try to use it against the production server, it fails. Somehow, Net::SFTP still insists on creating directories on localhost, and fails uploading the files because it’s missing these directories on the remote server. It’s as if Net::SFTP retained the memory of the old configuration, and I can’t shake it loose.

So I let it fail and check the stack trace, followed by a long session of grokking Net::SFTP code and a few more experiments to nail it down.

Turns out that Net::SFTP does not implement the mkdir method. Instead, its Session object waits for method_missing to catch the call and redirects the request to the underlying driver. Driver doesn’t implement mkdir either, again using method_missing to redirect the call to the underlying protocol implementation. There are five of those, one for each version of the SFTP protocol, so I go over them in order.

That part of the design is ingenious: how Net::SFTP behaves depends on which version of the protocol it negotiated with the server. It’s a lightweight plugin architecture. But it’s not the source of my problems, mkdir is implemented since version 1 of the protocol.

The problem turns out to be the combination with Rake.

Rake assumes that most of your objects are rake task, or somehow deal with tasky stuff. To make your life easier, and boy is it easier to use than Ant, Rake throws a lot of enhancements into the Object class. You can define a task from any object, which means in any context. For good measure, it also lets you use FileUtils methods from any context, er, object.

And right there was the problem staring me in the face. Because Rake adds mkdir to any object, calling sftp.mkdir routes the request straight to FileUtils, and never falls back on method_missing. The code works locally because it creates the directories through the file system, not the SFTP protocol.

One of those bugs that pass with flying green in the test cases, and fail miserably in production.

Once I figured it out, it was easy to solve. I monkeypatched Session and Driver to implement the mkdir method that would simply forward the request straight to method_missing. One meta-programming trick to solve another.

Try it yourself

Picking a host (anything but localhost) and a path you can access, try the following from a regular Ruby file:

Net::SSH.start(host) do |session|
  session.sftp.connect do |sftp|
    sftp.mkdir path
  end
end

Next, try running the same code from a Rakefile, and watch as it creates the path on the same machine.

To prove that the fix works, add this (stubbing only Session or only Driver will not work):

class Net::SFTP::Session
  def mkdir(path, attrs = {})
    method_missing :mkdir, path, attrs
  end
end

class Net::SFTP::Protocol::Driver
  def mkdir(first, path, attrs = {})
    method_missing :mkdir, first, path, attrs
  end
end

Peoplepatching

It’s great that I can monkeypatch Net::SFTP to do what I want, but more interesting to me is learning how to not inflict the same problems on my own code. So I came up with some lessons learned, and easy rules to follow.

Three for now, if you have more to suggest, I’m all ears.

1. Use method_missing sparingly. Defining methods is easy enough, so do it as often as possible, and let method_missing deal with hardcore trickery. As a general rule, if you can’t look at the code and figure out which methods it provides, reconsider how to implement them.
2. Use declarative constructs. Unfortunately, Ruby lacks in this department, it’s sometimes impossible to tell how classes and objects are extended. On the other hand, Rails, Facets and other projects have meta-programming methods that are much more descriptive and meaningful. Use those as often as possible. Where Rake redefines Array methods into FileList, it would be better if it included ActsAsArray.
3. Document that which you extend. And no, burying the text somewhere inside the code doesn’t count as documenting. You want it up there at the top of the RDoc, where your eyes are scanning the list of defined and inherited methods.

Now, excuse me while I go and check my old code over again, and get rid of some rogue method_missing and patch the documentation.

_{Photo by michael gallacher}