Downtime, Updates
Bob McWhirter
17 November 2008

Throughout the day (I've scheduled a daily maintenance window from 9am-5pm US Eastern every day...) this site occasionally will go offline while I update the code behind it.

Sometimes I'm updating the Rails application that powers it.  Sometimes I'm updating the Rails deployer that lights it up.

Application Updates

In the case of just updating the application, the downtime should last all of 5 seconds while AS redeploys the app atomically after Capistrano has successfully updated the code and such on the server. 

Capistrano checks the code out from the repository (GitHub) into a timestamped directory.  It creates a symlink named current pointing to the newly deployed directory of code.  It then emits an updated -rails.yml deployment descriptor in the JBoss deploy/ directory.  Every few seconds, AS scans the deploy/ directory for updated things to deploy.  Seeing the new -rails.yml, it first completely undeploys the old application from the old timestamped directory, including any scheduler.  Then it deploys the new application from the new timestamped directory.

We intentionally do not use the current symlink when deploying to AS.  If we did, as soon as the symlink was created, the code would be visible to the "old" version of the running Rails application before it gets completely undeployed.  This could result in errors if you require model migrations, changes to config/** files or have added gems. 

By using the timestamped directories, we can ensure that the old version of the application remains intact until fully undeployed.  Then it gets atomically replaced by the new version.

Deployer Updates

I have yet to automate updating of the deployer itself, but I will soon.  I update the deployer itself every day or so, after integrating new features or improvements (thanks Ales!).

To update the deployer, I bring AS down completely (cap deploy:stop), pull the new code from GitHub, run maven, and restart AS (cap deploy:start).  On this server, I've just symlinked the maven build output over to my AS deployers/ directory.

All told, updating both the deployer and the app (including ssh'ing over a very-high-latency satellite connection) takes right about 10 minutes.

Bottom Line

The update this morning brought about changes to both the deployer and the app. 

The deployer itself now uses Microcontainer slightly better, thanks to code reviews from Ales Justin (Mr. Microcontainer). 

And the Atom feed should now include better URLs for links in the entries.  Thanks to Adam Warski for noticing and reporting that.

  • Next Overview Presentation
  • Previous Cron in the Container
Copyright 2008-2010 Red Hat, Inc.