1. Dec 2nd, 2005

    Extending database support in reliable-msg

    I’m looking at ways to add more database support to reliable-msg. Currently thinking about using the Rails database adapters.

    ActiveRecord has an ORM layer, which I’m not going to use, since the SQL statements are few and easy to write, and I generally like to know what SQL statements I’m executing against the database.

    ActiveRecord also includes a layer of database adapters, with a single interface that can talk to any number of database drivers. That helps when you want to write the code only once. Currently it supports SQLite, MySQL, PostgreSQL, DB2, Oracle and SQL Server. So just pick which database to use from the configuration file.

    You don’t need to be running Rails, ActiveRecord can be used on its own merit and in standalone applications running outside the Web server.

    The downside is adding another dependency, which shouldn’t be a problem for anyone using Gems. At any rate, I expect we’ll see a few more Gem dependencies in the future, and more projects importing ActiveRecord. It’s a great way to build modular applications.

    Posted by Assaf Filed in general 6 Comments »

  2. Dec 2nd, 2005

    Change to message payload in reliable-msg

    The next release of reliable-msg will introduce two changes to the API. I hope it won’t cause much of a problem.

    The first change is in naming. Messages will still have identifiers and headers, but the meat of the message is now the payload. This just removes confusion in the API and documentation. Retrieving a message will return an object with the method payload, and the misnamed method message will be deprecated.

    The second change deals with the payload type. The next release will enforce the payload to be a string and will no longer accept arbitrary Ruby objects. There are two reasons for this change.

    Using Ruby objects limit the protocol to DRb, since we’re passing around data that only Ruby understands. I want to add HTTP support, be able to exchange messages with non-Ruby applications, maybe use stomp. I can do that with strings.

    Second, to retrieve a binary message the code must be able to unpack it. There are some cases where the code handles the message but doesn’t act on its content: move it around, archive it, etc. If the object needs to be unpacked, you need all the Ruby classes used when creating the object, not just the base Ruby library, which makes it impossible to write generic utilities.

    Quick Workaround

    Well, may not a workaround, because it lets you do more, if you choose to. Say you want to sends some arbitrary Ruby object asynchronously. You need to marshal and unmarshal it yourself, but Ruby makes that dead simple. To send the message:

    queue.put Marshal::dump(object)
    to extract the message
    msg = queue.get
object = Marshal::load(msg.payload)

    And if you want you can also replace Marshal with YAML and dump/load into a YAML document that’s more portable, or use XML, or even JSON.

    Posted by Assaf Filed in general No Comments »

  3. Nov 26th, 2005

    Integrating Reliable Messaging with Rails

    Talk about easy. The new release (1.1.0) adds two new method to your controller. Use queue to create a new Queue object for putting/getting messages in queue, or topic to create a new Topic object. Here’s what it looks like in action, a simple action that puts the message ‘Hello world’ in a queue:

    require_gem 'reliable-msg'

class ApplicationController < ActionController::Base

queue :my_queue

def index
my_queue.put "Hello world"
render :text => "Put message 'Hello world' in queue"
end

end

    And if you’ve just noticed, the new release adds topics for pub/sub style of messaging, and new and improved selectors.

    Posted by Assaf Filed in general 1 Comment »

  4. Nov 17th, 2005

    Reliable Messaging for Ruby

    63760739_42e72b9ed1[1].jpg

    Reliable What?

    Reliable messaging and queuing for Ruby. A way to exchange messages asynchronously, and build systems that scale better.

    • Simple API: put and get.
    • Transactions for dealing with message processing errors.
    • Best effort, repreated and exactly once delivery modes.
    • Selectors for processing specific messages.
    • MySQL or file system for message store (or write your own).

    Because …

    It’s easier to build loosely coupled systems if you can queue a message from one place, process it in another, and not require both parts of the system to be running at the same time.

    • Responsive UI: Perform lengthy operations in the background instead of having the user wait.
    • Balance workload: Break heavy processing into smaller tasks, queue them, and distribute processing across servers.
    • Mask failure: Simpler failure handling when dealing with disconnected and unreliable services.
    • All the cool kids have it.

    Last seen at

    You can download the Gem/sources form RubyForge

    The documentation

    SVN (living on the edge)

    And watch this blog for upcoming posts and releases.

    Image by ehecatzin

    Posted by Assaf Filed in general No Comments »

  5. Nov 17th, 2005

    Reliable Messaging for Ruby

    Reliable What?

    Reliable messaging and queuing for Ruby. A way to exchange messages asynchronously, and build systems that scale better.
    Simple API: put and get.

    • Transactions for dealing with message processing errors.
    • Best effort, repreated and exactly once delivery modes.
    • Selectors for processing specific messages.
    • MySQL or file system for message store (or write your own).

    Because …

    It’s easier to build loosely coupled systems if you can queue a message from one place, process it in another, and not require both parts of the system to be running at the same time.

    • Responsive UI: Perform lengthy operations in the background instead of having the user wait.
    • Balance workload: Break heavy processing into smaller tasks, queue them, and distribute processing across servers.
    • Mask failure: Simpler failure handling when dealing with disconnected and unreliable services.

    Last seen at

    You can download the Gem/sources form RubyForge

    http://rubyforge.org/projects/reliable-msg/

    The documentation

    http://trac.labnotes.org/cgi-bin/trac.cgi/wiki/RubyReliableMessaging

    SVN (living on the edge)

    http://trac.labnotes.org/cgi-bin/trac.cgi/browser/reliable-msg/

    Posted by =ivanoats Filed in general No Comments »

  6. Oct 18th, 2005

    Ruby UUID Generator

    Because every identifier wants to feel special. When you’re building a system that moves data around between different components, one of the first problems to tackle are how do you make sure all identifiers are unique. You can use sequences inside the code, but identifiers stop being unique when the application restarts. You can use auto increment keys in the database, but that only works if the data is tied to one specific table. There are hacks like using pre-allocated sequences backed by a file, or a sequence table in the database. But all tend to get a little complicated when you want to not worry about how the identifiers are generated, or who gets used to them. Continue reading Ruby UUID Generator

    Posted by Assaf Filed in general 19 Comments »