Labnotes

Tumblr is my latest fave. It’s a form of light blogging, a mechanism for filtering your feeds to the rest of the world (or the couple of people who care). It’s somewhere in between the thoughtful blog and the lighthearted Twitter.

Today, they added two new cool features. u can pst txt n pix v sms.
And you can add other tumblrs to your friends list, which shows their posts in the dashboard. Unfortunately, they don’t yet have an aggregate feed of all the people I’m stalking, and you can’t add friends who use custom domains, so don’t expect it to work with bytesized.

I also just found out you can twitter your tumblr and tumblr your tweets, and increase blog pollution even further by pulling straight from your Flickr or del.icio.us accounts.

It’s no secret I have a love/hate relationship with Maven 2.0. I hate it. And I love using it as an example of how not to build software. Maven 2.0 is proof that good intentions, hard work and an open source community can still lead to all kinds of wrong.

And apparently I’m not the only one who feels that way.

It has come to my attention that you are responsible for the loss of many, many hours of my life, hours that could have been spent on more worthwhile endeavours but that now I will never get back. In addition, the resultant fluctuations in my blood pressure have likely reduced my life-expectancy by a corresponding amount.

– The Fishbowl: An Open Letter to Maven 2

The road to Maven started full of hope and promise, quickly adding one module after the other. But in less than a month the build escalated into a maintenance nightmare. I seriously could not believe spending a whole day on the trivial task of copying one file from here to there. Can you imagine it takes 34 lines of XML code to concatenate two SQL scripts into one? And that’s before we start using them to populate the test database.

So everyone around me is wondering “what’s taking you so long?”, and I’m asking myself the same question. Did I finally meet my foe? The one technology that’s groundbreaking, forward thinking, and just beyond my grasp? Is it time to admit defeat and move on to something else. Maybe landscaping?

But I stayed the course, got something half working, enough to get the other developers to start using the new build. I’m glad I did. Now that everyone else was in the same sinking boat, I no longer had to explain the unexplainable. I finally felt vindicated. It wasn’t me. I wasn’t a retard, just unfortunate enough to use the build system that couldn’t.

We were already committed to make a release, and so the project went live, Maven and all. But we knew what will come next. The continuous build server reported random bugs, sometimes it would build, sometimes it won’t. Every few days, a developer would e-mail to let us know the build system doesn’t build. When it just did, a minute ago.

We quickly dubbed it the Maven Uncertainty Principle, though in fairness we knew there was little uncertainty or doubt whether the build will work: it won’t.

Almost everything I’ve ever downloaded that used maven for its build process, didn’t build.

– Tomcat Scales to 16,000 Concurrent …

Finally, one day we just had enough. The test cases stopped working. The surefire plugin surely did not fire. Even worse, a few days before I had to go and disable one of our critical test suites. Maven couldn’t deal with the fact that we excluded an old Log4J brought in by one of the dependencies, so we can then include the Log4J version we certify against. Perhaps Maven was telling us not to use test cases and we were too thick to listen.

Either way, we had to choose. Agile or Maven. One of them will have to go.

So we started imagining the happy life we could all live with Maven out of the way. Maybe we’ll switch back to Ant? For all its shortcomings, at least Ant builds work. Repeatedly. With certainty. You tell Ant what to do, and it does exactly that. No more guessing, hurting knuckles from crossed fingers.

Sure, Ant is not “declarative”, the holy grail of software developers everywhere. But declarative done badly is worse than any imperative spaghetti code. Between the phases, goals, profiles, modules, transitive dependencies, configurations, other people’s POMs, plugins, build.xmls all over the place, and the funky reactor, how do you know what your build will end up doing? It requires running a lot of imperative code. In our head, and by trail and error builds for hours on end.

With Ant, we just had to imagine and make it happen.

Or maybe we should try something else? How about Rake, the Ruby equivalent of Make? It lets you declare tasks and handles the dependencies between them. It slices through file dependencies like a hot knife on butter. It even has a Make-like rules system. It uses Ruby which, we’re told, is more capable than the Ant mock-language. It has … get ready for this … mutable variables, functions and classes! We can use these to keep our code DRY and neat, write and reuse code. Heck, we could even write “plugins” with just a few lines of code

So we did it. We started with Rake. Let me spoil the surprise for you. It turned out to be an excellent choice.

Rake was a good start, but not what we’d hope for. The typical Java application we work on consists of several modules, all of which have the same common lifecycle tasks: compile, test, package, deploy. Writing those over and over for each and every module would make it not much better than Ant. There’s got to a a better way.

They don’t just all have the same lifecycle tasks, they also have the same directory structure. So we used that to build a little framework that, by just pointing at a project could infer all the common tasks based on the files it finds. And for everything that deviates from conventions, we could just script with a few lines. There’s no escaping all these edge cases and special considerations, but with Ruby we can make them easy.

Want to see some examples?

Build me a JAR:

define "utils" do
  compile.with COMMONS.logging, COMMONS.pool, LOG4J, XERCES, JAVAX.stream
  package :jar
end

With XML beans generated from the schema, if you don’t mind:

define "bpel-schemas" do
  compile_xml_beans "src/main/xsd", "src/main/xsd/*.xsdconfig"
  package :jar
end

No, wait, I want to merge two SQL files I just generated, and build a database from that:

derby_sql = concat("derby.sql"=>[engine_sql, quartz_sql])
derby_db = Derby.create(”derby/jpadb”=>derby_sql)

The nice thing about the little example, is that it will build the database once, but regenerate it every time you chance one of the source files. There’s a free t-shirt for you if you can show me how to implement this in two lines of Maven or Ant code.

Now, let’s compile, package and install all of that in the local repository, so we can use these modules in other projects:

rake install

That, in a nutshell, is how we went from the deeps of frustration to a build system that … hold on a second … is actually fun to use. And dead easy to customize. And works repeatedly.

Say goodbye to frustrations and the Maven Uncertainty Principle. Stop arguing over declarative vs imperative, just have both.

Go ahead, give it a try.

Now I hate using the Beta tag, but let’s be clear, we’re a few short weeks from making a final release. We’re using it in several projects already, enjoying every minute of it. We’re happy, the continuous integration server is happy, life is good. But it’s still rough around the edges, and some bits will change before we hit 1.0. And the documentation is terse.

So if you want to get started today, I recommend picking up an existing Rakefile (and check out some of the custom tasks). It’s a complex project, but it will show you the extent of what Buildr can do.

Ismael proposed an interesting test yesterday. Suppose that the Web of Services really takes off. Not in the sense of selling more passes to conferences, issuing hourly press releases, or banking on product upgrades. Take off with businesses actually looking at services they can reuse based on availability, functionality and associated policies. A world where the semantics of SOA apply in practice.

In this hypothetical world of reuse and composability, people are going to turn towards that which works, is easily reused and commercially viable, so they can move on to solve higher level problems using existing solutions. They’re going to rely heavily on services providers, the SalesForce, Amazon and Google of this world.

I did say it was hypothetical.

But bear with me. So now we’re in this twilight zone where developers spend more time reusing existing services than building new ones. Practicality happens often, and outweighs any discussions over the injustices of WS-ResourceTransfer. The most transferrable skills, the best return for your code, comes not from where vendors are pointing you, but where existing service providers are.

And there lies the test. Imagine, if you will, that world of rampant reuse. Of a Web of Services that crosses networks, applications and use cases. In your opinion, which Web technologies are more likely to mature and survive?

Bill de hÓra (via Pete Lacey):

I think we need to talk about how “REST as in the HTTP Web, sucks”, before it becomes another silver bullet

I used to naively think that if I inspect assumptions, put things in perspective, and review them once in a while I’d be doing good for me and those around me. I still feel that way. But I long learned that those who want to believe a silver bullet exists do not tolerate such foolishness.

Let’s have this discussion for our own good, and by “our” I mean whoever cares for the ins & outs and using this hammer strictly for nails.

Everything old is new again. Mathew Ingram found evidence that the Semantic Web predates to 1910:

This new research environment would do more than just let users retrieve documents; it would also let them annotate the relationships between one another, “the connections each [document] has with all other [documents], forming from them what might be called the Universal Book.”

Unfortunately, “The Mundaneum moved into a series of smaller spaces, and eventually took over a parking garage before closing for good in 1934″.

Towards a better Web. Defective Yeti suggests Internet Access Captchas:

And so, a modest proposal: Internet Access Captchas, built right into browsers, designed to greatly reduce the overabundance of youtube commenters, MySpacers, and bloggers.

We can even dial it up and down, from the correct 3-letter spelling of you, to sentences that may contain whom.

Public Service Moment. The periodic table of the SOAPs will help put some order in your WS-Complexity. Includes all the relevant (and non-) specs, the dependencies between them (illuminating), current version number (higher numbers mean something) and the vendors behind them (enough said). (Via Bill de hÓra)

In JSON we trust. So was it really an AJAX exploit or just a media exploit?

The first egregious dishonesty in their paper is that it analyzes client-side Ajax frameworks. This is not where the vulnerability is, nor where it’s fixed. … That’s right, the simplest fix is to always make sure you send an object on the outside, not an array.

Haskell makes it to TechMeme. Or more precisely, a discussion that’s pure software engineering and development practice. It breaks the natural order of things which, as we all know, must flow from tech politics to latest scare to celebrity gossip. Just like the 10 o’clock news. Still, I like it. Way to go, Reg.

So the system went live, and the requests are pouring in like crazy, and the phones are off the hook with angry customers on the other line. “Where’s the product I just ordered?”

“Let me get back to you.”

Turns out it was a big mess. Customers were sending orders in broken messages we couldn’t understand, we couldn’t even tell where to send our errors to. We had orders with no item identifiers, and orders with items inside the shipping address, and orders with zipcodes instead of phone numbers. What a mess. One order looked like it was CSV, another content encoded as a JPG.

Joe said it will only take a couple of weeks, but we can get the server to check all incoming orders and send error responses. Management wasn’t happy. Then Bob chimed in, said it will only take a day to create a schema and run a validator on all the inputs. Guess what we did next?

No more angry customers. In fact, customers are deligheted judging by the sudden increase in order volume. Then we get a call from the CEO. How come sales are up, but we’ve just lost $10m? We check with accounting. Everything matches up. Check with the bank. No fraudulent transactions. Run a report on the sales system, get back the same data. All’s well, but something is a miss.

So we pick a few customers at random, and run through their orders from input to output, and we find it. It turns out some customers managed to game the system. They sent us orders that looked like that:

<order>
  <item>MacBook Pro 17"</item>
  <total currency="USD">65</total>
</order>

Oops. We listened to Bob. We checked the markup, but we forgot to check the data.

Does the world need another markup definition language?

(There’s a lot more I can say about XSD pretend-extensibility, the namespace illusion of versions, or what happens when all you do is exchange semanticless WSDLs and prey for interoperability, but I feel one point is enough)

I think the root of the problem is that we’re trying to find an abstraction from the implementation, only to realize — the horror — that no abstraction is as good as the implementation it attempts to replace.

We can’t all agree on the one development language to use, some camp with Java, some swear by Ruby, some know their Python and some bought into C#. So clear what we need is another language-neutral language. Like XSD. Yes. No, maybe RELAX-NG. Sometimes. But really, Schematron.

At some point that need to not deal with languages ends up creating more half-backed languages than we already need. We’ve substituted inevitable complexity for unnecessary complexity.

Every blog has its own rhythm. I believe there’s some common wisdom about defining your target audience, content guidelines and writing style. In my case I just started posting and the rest happened. But every once in a while you need a change of pace.

As a regular reader, you would expect opinionated posts about the REST/WS War of Attrition, the dynamics of meta-programming, and my aversion to Interchangeable Developer Encapsulation and “tooling”.

Recently, I tried adding some flavor to the blog, with a handful of short tech-less posts on whatever I felt like talking about, and … well it didn’t stick. It felt off, different, conflicting.

So I stopped and went back to the mythical drawing board. I tossed a few ideas in my head, but eventually all roads lead back to Tumblr. So I decided to give it a try. I’ve seen a few tumblelogs before and liked the format, but never seriously considered them. Well, add that one to the list of eye opening experiences.

I won’t try to define a tumblelog, just tell you what I like about it. The format is short posts. No archives, categories, sidebar, not even a way to leave comments on individual posts. Lightweight. These are not conversation starters, they’re pointers to interesting things that happen around us. Links to other sites, cool photos, memorable quotes, snippets of consciousness.

We rarely consider how much software design influences the way we operate. It promotes that which it makes easy and accessible. And I’m not above being influenced. I like the editing facility of WordPress with its WYSIWYG editor, but that which makes it infinitely flexible and capable, also renders it an obstacle. It’s too slow.

It takes me too long to format quotes, hyperlink text, position images and double-check that paragraphs don’t run into each other. That’s a filter. And a lot of short posts get filtered into oblivion.

Enter Tumblr.

Tumblr is a lesser known service that deserves much more attention. In two wrods: it rocks!

Tumblr has six types of posts: links, quotes, photos, video, conversation snippets, and anything-goes HTML. It’s that distinction of content types that makes posting so damn easy. Click the bookmarklet and pop comes a window for linking to the current page, just hit submit and you’re done.

Worth quoting? Select the text and click the bookmarklet for an insta-quote. Interesting photo? Hit the bookmarklet and you’ve made a post. Did I mention damn easy?

The pop-up window lets you tab between different post types, so you can go from quote back to link, or from link to photo (it even lets you pick an image without hunting & gathering the URL). Switch between the tabs and you’ll understand that there are nuances for each of these post types. Treating each type differently, and the single-minded focus on quick/short posts is the secret that will turn you into a speed-blogger.

So there.

Give a warm welcome to bytesized.labnotes.org, the lighter side of Labnotes. Just enough daily posts to help you procrastinate, or keep you company while you’re waiting for Eclipse to load up.

As for me, apparently I’ve been relegated into a machine for moving interesting posts from my feed reader to yours.

Update: The brains behind Tumblr are planning new features, which earned them a favorable writeup on TechCrunch earlier today.