He Who Controls The Spice

Hal Hildebrand about the politics of JSR 277, Sun’s weapon against the NIH OSGi spec:

And there in lies the nub of the problem. As has been said by many before me, “He who controls the Spice™, controls the universe”. And what is quite clear from the conversation with Alex is that Sun believes it controls the Spice™. And it’s quite hard to argue with this fact. One can look at the dead bodies strewn along the path of any number of previous JSRs and see how Sun dealt with technologies that predated the JSR in question, inspired the JSR in question and were gutted and left to die in the hot sun by the JSR in question. Clearly, one can look at the writing on the wall and see much the same in the near future of OSGi. Once something is in the JRE, any technology competing with it is toast. End of story.

Good read and pretty relevant. OSGi is not the first or last work threatening a Sun technology and being subjected to the JSR treatment. Notice that Sun positions OpenJDK as ”under terms compatible with Open Source ideals”, without ever clarifying: ideal to who?

Open Development Ideals

If you didn’t get the chance yet, check out Roy Fielding about Open Architecture (PDF). He gave the presentation at OSCON, based on the Open Architecture Software: A Flexible Approach to Decentralized Software Evolution (yeah, PDF), a dissertation by Peyman Oreizy’s.

You’ll want to read these two first before continuing. Actually, if you’re short on time just go and read these two. The focal point is Conway’s law:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.

No matter how much we protest in denial, communication structure is an arbitrator of software design. That makes open development an architecture of software created out of the desire for a particular communication style: less.

Not to be confused with lack of communication. Open development is very chatty, and it uses tools that amplify that (mailing list, blogs, wikis, and such). There’s a lot of communication, but very little of it is forced by the structure. The necessary communication to experiment, create or modify a feature is kept to the bare minimum by keeping most features outside the core.

The basic principles for open development as laid out in slide 17:   

What is common to all of the largest and most successful open source projects?   

• a software architecture 
• designed to promote anarchic collaboration 
• through extensions 
• while preserving control over the core interfaces

I think these four are spot on.

OSGi is one of these open architectures, it does little more than offer a core interface around which you can build any number of extensions. Eclipse is one big set of extensions written around that very small core. Other open architectures listed in the presentation are Apache, Linux, Emacs, Firefox and Sling. I’m going to add three more which are necessary to make the point: Git, REST and Ruby.

Open architectures are very modular, modules simplify the core, enable independent development and promote experiments. Looking at it from the perspective of the project (what’s in it for me?), you gain reduced friction, anarchic growth, more features and less communication. The winning combination.

Those were a few more bullet points from the presentation. With me so far? So let’s see what’s special about Git, REST and Ruby.

Git, REST and Ruby

I added Git because I think it typifies open development even more than, say, Firefox. I’m not excluding other DVCS, I just got more chance to work with and explore Git. Git has a small kernel around which you’ll get a hundred plus command line tools. That’s just the default setup. You can keep adding features on top of that kernel, a quick search leads me to gitosis, git-issue, gitbak, git-wiki, gitjour. Modular architecture never takes the day off.

But modules are only half the picture. Think of the way we use Git, not the design of it, but the development practices it promotes. It enables independent development and promotes experiments. There’s less friction and less communication required: if you need a branch to start hammering a new feature there’s no need to contact your local Write Access Bureaucrat. Its distributed nature is often compared to anarchic.

I added Git because it preaches and enables the very same principles it’s built upon.

I added REST to that list because the renewed interest comes from the very same place, an open architecture for Web services. The uniform interface, minimal content types and other constraints help preserve control over the core interface, at the same time allowing everything else to happen with less communication and friction.

REST is interesting in that it captures the best practices of the Web, and the Web emerged out of open development. The biggest of its kind.

Why Ruby? To make a point about social aspects that I can contrast with another well known language. Ruby has a modular architecture (Gems), which is not particularly different from other languages or platforms (Debian, PEAR, OSGi, etc). The majority of development happens outside the core, but occasionally some of it  would get absorbed into the core, when it’s deemed useful and popular. For example, Ruby 1.9 absorbing Gems and Rake.

The Spice Must Flow

In Javaland most of the interesting and important developments happens outside the core, but the process breaks when they become de facto standards and start threatening the core. The core is not the community, but a company, and the prize for being widely useful and popular is elimination.

Log4J was a necessity, core fought it with javax.logging; XML Apache was an open source milestone, abstracted away by JAXP; Hibernate is the people’s choice, that’s why we have JPA; OSGi is reaching escape velocity, and here come the JSR277 troops. I recently heard the scepter of success is looming over Spring.

To be fair, Sun is in good company. Microsoft punishes its open source ecosystem the same way, Apple’s communication structure is the antithesis of open. All three are busy fighting open development, which they consider subversive to their business goal, or just the way they like to do things around here.

Large vendors work that way because they take decision and then inflict them on others. They listen, only so far as acknowledging Gesundheit when you sneeze is considered listening. It’s a one-way flow. Of course, “patches are welcome”, which gives the illusion of community collaborating around shared goals, but patches are welcome only when they fit preconceived business decisions.

You’re right to point out that none of this is new, but I’m sensing the friction building to a scale we haven’t seen before. It will be interesting to revisit this five years from now and look at the progress we made. Are vendors ever benevolent dictators? Will the community punish them for pushing their own agenda and subverting open development? Will it even make a difference?

Closing

Next time you find yourself treading in open source waters, consider that success often comes from embracing, not subverting the open development architecture.