Async Task Queues: Ruby meets JMS
Bob McWhirter
31 March 2009

I keep having "asynchronous messaging" listed in my presentations as something else "beyond rails" JBoss could bring to the Ruby table.  A guy (dude, sorry, I forgot your name) at DevNexus got excited by that idea.  So I've been hacking.

Instead of just bridging JMS to Ruby and saying ta-da, I noticed that most folks in the Ruby world currently play with async task queues.  Hang out in #github long enough, and you'll see them talking about pausing the queue, restarting the queue, the queue being clogged, etc. 

Here, they're using the queue not so much to pass messages, but to trigger a unit of work to be executed outside of the "normal" thread of control. 

Sometimes that thread is a web-request, and they queue up an action like forking a repository.  Sometimes that thread is a post-commit hook, and they queue up actions such as IRC notifications and other integrations.

Thus, I've started to implement JMS, but so far only exposing it as an asynchronous task-queue.

I'm completely open to suggestions, but my current musings think we want a ruby class per queue.

class PostCommitHookQueue
# I'm a queue!
end

Maybe each queue can accept a handful of different tasks to execute.  Let's represent these by methods.  Tasks certainly need some parameters, so let's have a payload argument to contain those parameters.

class PostCommitHookQueue

def generate_commit_mail(payload)
# generate an email containing the commit
end

def notify_irc_channels(payload)
# go ping the project's IRC channel
end

end

We create this under app/queues/post_commit_hook_queue.rb, and we're off and running.

But how do we send stuff to it?  I'm thinking of a class method enqueue(task, payload).

Here, a client is adding a :generate_commit_mail task to the queue, with a hash of parameters as the payload.

PostCommitHookQueue.enqueue( :generate_commit_mail, 
{ :project=>'jboss-rails',
:hash=>872e971ea8 } )

This call is non-blocking, as it's ultimately just sending a message to a JMS destination. 

Under the covers, each Ruby queue class ends up deploying an actual JMS queue, which can be administered like any other JMS queue within JBoss.

Once we get this going, I'll look into how we can expose clustered queues, durable queues, and all the other functionality of JMS.  But do it nicely using the semantics of this async task queue use-case.

Any thoughts on the design?  Any suggested use-cases for one-to-many topics, instead of just one-to-one queues?

  • Next Throwing objects to tasks
  • Previous JBoss-Rails 1.0.0.Beta5 released
Copyright 2008-2010 Red Hat, Inc.