
Somewhere in my ever expanding list of drafts I’ll never get to finish is another post about the economics of Ruby, and how raw performance is less of a problem when you’re bound by the database, spend less on development, and can optimize in the large. Basically, regurgitating the same justifications I used to explain Java a decade ago.
But this is not that post.
Today, I’m going to talk about something else, and share with you an interesting discovery from working on Buildr. There will be no language theologies or abstractions of performance, just the facts.
It started when Maven hit the fan, and we had enough of the clunky XML pseudo-code, unreliable builds and maintenance straight out of Elm street. So we replaced Maven with Buildr, a build system written in Ruby. To make our life easier, we designed Buildr to be a drop-in replacement for Maven. We’re building the same code, running the same tests, compiling the same XMLBeans, creating the same Hibernate schemas, sharing the same remote and local repositories.
All this to say, they’re black box equivalent. Feed them the same project, and they generate the same JARs, WARs and distro files.
But they’re not entirely equivalent.
Off the bat, we downsized 5,443 lines of XML abuse spread over 52 files, into a single build script weighting a measly 485 lines. It’s amazing what a real language, with proper variables and (gasp!) functions and objects, can do. Now that our build is down to a healthy BMI, it even looks sexier.
It works repeatedly, and works like we expect it to, so we spend no time fighting the build, and more time writing new code, fixing test cases and anything else we’re supposed to be doing. Let me tell you a secret: we even get to go home earlier.
But that’s not all.
Ladies and gentlemen, the moment you’ve all been waiting for. Where we get to talk raw horsepower and 0-60 times. But before we get to numbers, let’s explain what they mean.
Remember, we’re using Ruby, not exactly the fastest programming language. We’re using it to build Java code, so we run the Ruby VM alongside the Java VM. There’s a lot of dependency management going on, you need that for reliable builds. And, when it comes to implementation, we chose clear and maintainable over fast and furious.
So let’s put expectations in perspective. “Fast” means not much slower than Maven. The question is, how close did we hit our target?
[assaf@casper ode]$ time rake clean install test=off real 1m1.827s user 0m38.228s sys 0m3.464s
[assaf@casper ode]$ time mvn install -Dmaven.test.skip=true -o real 1m58.082s user 2m23.287s sys 0m10.160s
Swooooosh.
Buildr does a full build at 50% the latency of Maven. Twice as fast.
Consistently.
Let’s try something else:
[assaf@casper bpel-runtime]$ touch src/main/java/org/apache/ode/bpel/intercept/ThrottlingInterceptor.java [assaf@casper bpel-runtime]$ time rake build test=no real 0m5.340s user 0m4.612s sys 0m0.534s
[assaf@casper bpel-runtime]$ touch src/main/java/org/apache/ode/bpel/intercept/ThrottlingInterceptor.java [assaf@casper bpel-runtime]$ time mvn clean compile -Dmaven.test.skip=true -o real 0m14.740s user 0m24.684ssys 0m0.649s
I measured full builds and partial builds, compiled different modules, measured downloads and uploads, test cases and what not. In every single case, Buildr performed as well as, or faster than Maven. Buildr flew through the partial build in 6 seconds, then sat there waiting a full minute for Maven to catch up!
Wow.
Of course, we’re not measuring raw Ruby against pure Java. We’re comparing one implementation against another, where they both do the same thing. Black box equivalent. That’s a real life benchmark.
Language speed tests are as relevant as promises made by politicians on election day. What really matters is the type of solution you can build, the effort it takes to build and maintain it, and how well it behaves.
We know the Ruby-based solution performs significantly faster, is much more reliable, requires less work to use and maintain, and took all of 3 months from concept to working release.
Ruby might be slow, but what you build with it can be devilish fast.
(*) All tests run multiple times, to make sure we get repeatable results. Test machine runs 2GHz Duo Core 2, Fedora 6, JDK 1.5.11 and Ruby 1.8.5. The non-trivial test case.
Photo by xxxtoff
Bravo. Great post, great job. Almost makes me wish I used Maven so I could replace it with Buildr. Almost.
You could probably run Buildr on JRuby and it would be faster than maven.
When it comes to Maven, you pretty much had me at ‘Ruby’. Very cool.
Where can we get a release version of Buildr with sample usage docs? Or are the rubydocs all we get? I am looking forward to testing Buildr out and replacing maven2 for my current project at work. Cheers!
Right now, the RDocs are the documentation. Clearly we need to write a primer. Also, you’ll want to check out a build we’re using for ideas and influence:
http://svn.apache.org/repos/asf/incubator/ode/trunk/Rakefile
Dude, maven is better documented than buildr, and I woulnd’t take that as a compliment.
You’ve been InfoQ’d –
http://www.infoq.com/news/2007/05/fast-ruby-builds-with-buildr
I switched a bunch of projects over to Maven 2 and haven’t looked back…they work great and Maven’s transitive dependency mangement is AWESOME!! Now explain to me why I would write my Java build script in using poorly-documented Ruby?
Pingback: Bert Lamb » links for 2007-05-09
Don’t you think it’s like using Java to build .NET project? It seems wrong from the begining. It’s faster, yes.. but maybe it’s better to use the effort to change the way Maven works?
I think you should always use the best tool to solve the problem. You’re willing to take the steps to learn an XML pseudo-language to use Maven, and an XML pseudo-language to use Ant, and then try and write/test/debug builds in a combination of these two languages.
Why not choose an approach that uses a real language, one that can do a lot more than the limited XML dialects, yet express the same build in 10/th the effort?
Pingback: prozz’s blog » Blog Archive » Ruby researches
How do i do variable substitution in buildr?My project uses GMU M4 files and the M4 files do the variable substitution.These variables are defined in an XML file.The XML file will have these variables defined for each environment like QA,STAGE and PROD.These M4 files will create an actual file and replace the variables.How do I acheieve that using buildR?
Where can I find some examples?
Thanks,
Nagesh
Nagesh, the Buildr web site and documentation is at http://buildr.rubyforge.org. I believe what you’re looking for is the filter task — copy one file to another and do substitution on specific tokens.
Pingback: A Year in Cocoon
well, from test i have made, maven is 2 times *slower* than ant, so i guess buildr is slower than ant. and since ant has ivy now, it has module based dependency.
also, why choose ruby as the scripting language and not groovy? with groovy you don’t need to run two VMs , just one, you get all of ant’s code base for free, and since groovy compiles to .class files, i think the build logic itself will be faster. it has similar DSL concepts to ruby (no parenthesis, code blocks). did you look into Gradle (http://www.gradle.org)?
Ruby is a better all around scripting language. You get all of Ant for free, we use Antwrap and a lot of Ant tasks. The cost of starting the JVM is insignificant, which is good since for running tests you want to start a lot of these — Buildr like Ant and Maven forks by default.
Buildr is as fast as Make:
I’m involved in a project for compiling C++ code. I assume you’ll be happy to know that buildr is as fast as Make. I’ve currently tested this with 13 modules containing 195 cpp files (it takes ~20 seconds for both gcc and make). The makefiles are generated by a Maven plugin, so they do very little except for actual compilation.
The truth is that I’m still shocked that it performs so well.
Also, I’ve recorded the compilation commands in a file and executed the file: it took 29 seconds. This is because it was executed serially (where in buildr/rake i did the obj compilation in parallel)
Thanks, Ittay.