Oddthesis-logo

Nobody should need cron

Ultimately, we all want easy-to-replicate deployments of our apps.  Perhaps we're deploying on a cloud, and deploy often on new virtual hardware.  Perhaps we're just paranoid, and expect an earthquake to hit our datacenter.  Or maybe we expect to grow like the 4H club's monster pumpkin, and fully anticipate moving to new/different hardware soon.

Either way, the fewer moving parts and manual steps you have to follow, the better.

To me, cronjobs are the most-often forgotten step. They don't take up visible room on the filesystem.  They don't listen on a port.  They're easy to forget.  You can forget to install them.  You can forget to remove them.  In general, cronjobs annoy me.

But there's still the need to do things at certain times, or on a regularly occurring schedule.

With Rails apps, we can vendorize our dependencies.  With JRuby-on-Rails, we basically vendorize the whole language.  And with JBoss-Rails, we have a really nice and complete environment.  Java has tried real hard to allow you to deploy entirely within a container.  JBoss AS certainly provides everything we'd need.

Specifically, it provides Quartz.

Since we're running in AS, we're happy to write deployers if needed, and we have a scheduler already setup in AS, I think we can bake scheduled tasks right into application deployment.

Imagine $RAILS_ROOT/app/scheduled-tasks/blog_poller.rb.

It could have a fairly simple execute() method on it.

When your app is deployed, $RAILS_ROOT/config/scheduler.rb could be read to set up the quartz tasks. 

Cron's syntax isn't exactly warm and fuzzy.  I think common case is a recurring interval for a task execution.  Something like...

Scheduler.deploy do |schedule|
schedule.every( 30.minutes ).run( BlogPoller )
end

Simple enough. If parameters are needed, perhaps something akin to

Scheduler.deploy do |schedule|
schedule.every( 30.minutes ).run( BlogPoller ).with(
:number_of_retries=>3,
:notify_on_failure=>true
)
end

Some syntax to support concrete time-of-day or day-of-week should be doable.  It might be worth investigating the chronic gem.

And since we're handling the deployment, we don't have to wiggle these in strangely as part of the web-app.  A deployer will look for config/scheduler.rb, fire up a ruby interpreter to figure out what the schedule is, and then go deploy actual scheduled tasks into the Quartz service.  Scheduled tasks can be a first-class component like a Servlet, a message queue, or an EJB.

Deploy your app, your scheduled tasks get deployed too.  Undeploy, and the scheduled tasks get cancelled and removed likewise.

 

Comments

Paul Brown, 06:50pm UTC, 10 November 2008

I have seen the pain that forgotten cron jobs (or ignored warnings from cron jobs or botched environment/paths or...) can cause with migrations and upgrades, and it's not pretty. The idea of having a full system build as part of the deployment onto the app server is compelling.

John Smith, 01:35pm UTC, 11 November 2008

You might not have realized it, but there are tons of batch jobs that aren't written in Ruby. cron is still an important tool. ...and as a system administrator, we just love it when the batch jobs are on the application server so when we cluster it, they run multiple times on multiple boxes.

Bob McWhirter, 01:15am UTC, 12 November 2008

@John Smith--

The invisible hand of sysadmins does like to smack developers, I realize. And yes, cron is still a very important tool. But in the land of green-fields java or ruby development, I think we can route around it.

As far as clustered deployment, there are ways to prevent the multiplicity you note. I'm one who thinks you should be able to deploy "the app" on a heterogeneous cluster, and have each node do the right job. Some are focused on the web. Some on business logic. Others on async or scheduled jobs.

Brian Gilstrap, 02:39pm UTC, 12 November 2008

Trying to recreate cron in Java/Ruby is asking for trouble. Any real-world application will have activities which are hard or impossible to implement in Java or Ruby. They will be OS-specific, or low-level, or be needed to monitor the very container you are trying to run them from. They often need to be extremely robust (more robust than Java apps can be) in the face of very unusual conditions on the box (huge load, almost zero memory available, etc). Cron and OS-specific tools are well-suited to these situations. Java and Ruby are not.

I'm a software engineer who has been working with Java since 1.0 beta, and I find it a great development language (I'm less experienced with Ruby but it is nice also). But they are simply inappropriate replacements for cron.

Login to avoid moderation.

Sign up if you need an account.

Creative Commons License Copyright 2008. Odd Thesis by Bob McWhirter is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.