Somewhere in my ever expanding list of drafts I’ll never get to finish is another post about the economics of Ruby, and how raw performance is less of a problem when you’re bound by the database, spend less on development, and can optimize in the large. Basically, regurgitating the same justifications I used to explain Java a decade ago.
But this is not that post.
Today, I’m going to talk about something else, and share with you an interesting discovery from working on Buildr. There will be no language theologies or abstractions of performance, just the facts.
It started when Maven hit the fan, and we had enough of the clunky XML pseudo-code, unreliable builds and maintenance straight out of Elm street. So we replaced Maven with Buildr, a build system written in Ruby. To make our life easier, we designed Buildr to be a drop-in replacement for Maven. We’re building the same code, running the same tests, compiling the same XMLBeans, creating the same Hibernate schemas, sharing the same remote and local repositories.
All this to say, they’re black box equivalent. Feed them the same project, and they generate the same JARs, WARs and distro files.
But they’re not entirely equivalent.
Off the bat, we downsized 5,443 lines of XML abuse spread over 52 files, into a single build script weighting a measly 485 lines. It’s amazing what a real language, with proper variables and (gasp!) functions and objects, can do. Now that our build is down to a healthy BMI, it even looks sexier.
It works repeatedly, and works like we expect it to, so we spend no time fighting the build, and more time writing new code, fixing test cases and anything else we’re supposed to be doing. Let me tell you a secret: we even get to go home earlier.
But that’s not all.
Ladies and gentlemen, the moment you’ve all been waiting for. Where we get to talk raw horsepower and 0-60 times. But before we get to numbers, let’s explain what they mean.
Remember, we’re using Ruby, not exactly the fastest programming language. We’re using it to build Java code, so we run the Ruby VM alongside the Java VM. There’s a lot of dependency management going on, you need that for reliable builds. And, when it comes to implementation, we chose clear and maintainable over fast and furious.
So let’s put expectations in perspective. “Fast” means not much slower than Maven. The question is, how close did we hit our target?
[assaf@casper ode]$ time rake clean install test=off real 1m1.827s user 0m38.228s sys 0m3.464s
[assaf@casper ode]$ time mvn install -Dmaven.test.skip=true -o real 1m58.082s user 2m23.287s sys 0m10.160s
Swooooosh.
Buildr does a full build at 50% the latency of Maven. Twice as fast.
Consistently.
Let’s try something else:
[assaf@casper bpel-runtime]$ touch src/main/java/org/apache/ode/bpel/intercept/ThrottlingInterceptor.java [assaf@casper bpel-runtime]$ time rake build test=no real 0m5.340s user 0m4.612s sys 0m0.534s
[assaf@casper bpel-runtime]$ touch src/main/java/org/apache/ode/bpel/intercept/ThrottlingInterceptor.java [assaf@casper bpel-runtime]$ time mvn clean compile -Dmaven.test.skip=true -o real 0m14.740s user 0m24.684ssys 0m0.649s
I measured full builds and partial builds, compiled different modules, measured downloads and uploads, test cases and what not. In every single case, Buildr performed as well as, or faster than Maven. Buildr flew through the partial build in 6 seconds, then sat there waiting a full minute for Maven to catch up!
Wow.
Of course, we’re not measuring raw Ruby against pure Java. We’re comparing one implementation against another, where they both do the same thing. Black box equivalent. That’s a real life benchmark.
Language speed tests are as relevant as promises made by politicians on election day. What really matters is the type of solution you can build, the effort it takes to build and maintain it, and how well it behaves.
We know the Ruby-based solution performs significantly faster, is much more reliable, requires less work to use and maintain, and took all of 3 months from concept to working release.
Ruby might be slow, but what you build with it can be devilish fast.
(*) All tests run multiple times, to make sure we get repeatable results. Test machine runs 2GHz Duo Core 2, Fedora 6, JDK 1.5.11 and Ruby 1.8.5. The non-trivial test case.
Photo by xxxtoff
Bert Lamb » links for 2007-05-09
prozz’s blog » Blog Archive » Ruby researches
A Year in Cocoon