If you were unlucky enough to miss my presentation on JBoss-Rails yesterday at the DevNexus conference, you can grab the entire deck right here.

I'm happy to announce after quite a lag, there's a new version of JBoss-Rails available with lots of new features. Just in time for playing with it over the weekend!

This release sees a solidification of the Job Scheduling component, along with the addition of the Ruby SOAP Endpoints component.  The SOAP stuff also brings along support for cyrptographic keystores and XML-to-Ruby-to-XML databinding.

Additionally, we've upgraded the version of JRuby (1.2RC1 now) and removed a dependency on JRuby-Rack.

New and updated documentation now exists.

We've also created a "My first Rails App(tm)" walk-through to get you started on that weekend project of doing a mash-up between your sprinkler-system and Twitter.

All of the artifacts are available from our newly-minted Maven repository.

This includes:

• The complete binary distribution (~30mb)
• The standalone JBoss-Rails deployer jar (~30mb, drag'n'drop, you're done)
• The jboss-rails-support component (~7kb) for your Rails app (we also have instructions for installing this via git.)

As always, we're available to chat on FreeNode, on the #jboss-rails channel.

And you can join the jboss-rails@ mailing list by sending an email to

jboss-rails-subscribe@oddthesis.org

The archives are available through MarkMail.

If you have bugs or feature-requests, drop them on the JBoss-Rails Lighthouse instance.

Sources are available on GitHub:

• jboss-rails
• jboss-rails-support

Introducing jboss-cloud-support

The most important change in JBoss-Cloud project is moving its core to a external project called JBoss-Cloud Support. Sources are available, as usual, at github.

You might be asking what's the reason of that move? As you know we at JBoss are open, really open. If you want to generate appliances, not neccessary for Cloud, you can use only jboss-cloud-support with self created appliance files and all needed stuff for that. But you should know that we're focusing on JBoss-Cloud project, though.

If something isn't working for you in a "stand-alone" version of JBoss-Cloud, tell us about that. We're on IRC and we have also a mailing-list. You can create a ticket too (remember to add core tag — this will help a lot!).

Wizard improvements

Wizard has been completely rewritten. What's new? In one word: everything. Wizard shipped with 1.0.0.Beta2 has enabled only selecting appliance and output type. Now you can choose everything you need:

• Appliance to build,
• Disk size,
• Memory size,
• Output type,
• Network name (if VMware Personal or Enterprise output type is selected).

More, you can save an appliance configuration and reuse it later! By default wizard will store configurations in ~/.jboss-cloud/configs/ directory. You can override this by setting JBOSS_CLOUD_CONFIG_DIR environment variable, eg:

export JBOSS_CLOUD_CONFIG_DIR=/opt/jbc/configs/

./wizard

At start wizard will show all saved configurations. After selecting saved config there are three options:

• view — it'll print saved configuration,
• delete — it'll remove selected configurations from disk,
• use (default) — it'll build selected configuration.

To run wizard go to JBoss-Cloud home folder and run ./wizard. Note that it was renamed from appliance-builder.

Tip: you can exit from wizard any time by pressing CTRL+C!

VMware support changes

VMware support code has been completely rewritten, too. Most important changes are:

• Switch to SCSI disks for VMware Personal appliances (Enterprise has SCSI disks already),
• Got rid of some dependencies: python-virtinst and qemu-img are no longer required!
• Increase VMware appliances generation speed. If you have generated RAW image before, converting it to VMware images, both Personal and Enterprise will take not ~20 minutes as before, but < 5 sec, really, check it out!

When it will be released?

We're getting closer and closer to next beta release of JBoss-Cloud. Take a look at tickets assigned to 1.0.0.Beta3 milestone. We're going to push it in the near future, so be prepared to get latest and best Cloud out there!

But if you are impatient (don't worry, we like them), take a look at how to start page.

Why VMware?

I'm a VMware fan, really. They provide good, reliable virtualization environment for personal use and big business too. In the middle of 2008 VMware has announced a new product called ESXi. It's a hypervizor, available for free (but still not opensource), for everyone. ESXi can be a part of VMware Infrastructure (you can choose between old, fat ESX Server and new, light ESXi).

Unfortunately ESXi can be installed only on certified hardware. Officially. Many people around the world have created a white-box list of systems (mainboards, controllers, etc.) capable running ESXi, so you can pick up a good, cheap piece of hardware and have really good virtualization at home (or company).

Before I started digging in JBoss-Cloud I had a wish to run it on ESXi (and VMware Infrastructure 3 too as formats are the same), so making JBoss-Cloud on ESXi working was my priority (VMware personal support was already done by Bob).

VMware disk types

For appliance image creation we use appliance-creator tool from Thincrust. Te default output format of this tool is a RAW disk. It's great for KVM, but not so good for VMware, or rather I thought so before... But, in turn.

VMware uses seven (sic!) disk types. We can exclude types which are breaking a disk in 2 GB files. We don't need them as we are using good filesystems, right? We don't need also physical disks. After excludes, we have still three types; two for VMware Personal: monolithicSparse, monolithicFlat and one for VMware Enterprise: vmfs.

BTW, I'm using the unofficial VMware Personal and Enterprise terms to distinguish VMware products intended to personal, home use (Server, Fusion, Workstation, Player) and enterprise, datacenter use (Infrastructure, ESXi).

So, back to disks. What's the main difference between monolithicSparse and monolithicFlat? It's simple: monolithicSparse is growing as there is more data to store on the disk and monolithicFlat allocates all disk size at the time of creation. Another difference is that monolithicSparse has a embedded disk descriptor, monolithicFlat has an external descriptor file.

Thincrust appliance-creator has support for various disk formats. How it is done? It creates a RAW disk image and, if needed, it converts it to a specified format using qemu-img. As qemu-img has a wide support for various formats, appliance-creator is a very handy tool, worth taking a look (or two)!

VMware disk support is also included in qemu-img. Unfortunately only monolithicSparse disk type is supported by this tool and we need vmfs. Second glitch: created disk is a IDE disk. It's fine for VMware Personal, but VMware Enterprise uses only SCSI disks. As the disk descriptor for monolithicSparse is embedded, there is no simple way to convert it to a SCSI disk (at least I don't know any simple method).

We need to boot it, now!

I understood that if I want to have JBoss-Cloud working on VMware Enterprise I must do it another way. After some time spent on vmfs disk format I realized that this is... simply a RAW disk! So, really good news, as we have a RAW disk already!

In VMware environment a disk needs also a disk descriptor file. For vmfs disk type the descriptor is fortunately an external, simple text file which can be edited by hand. I've created one for my disk with all needed information (disk size, geometry, SCSI controller etc.). It wouldn't be possible without a great site from Ulli Hankeln. Ulli has described the vmfs descriptor format and created also a disk geometry table for every disk size in the world. Thank you Ulli, great job!

After first run I saw the grub menu. I was happy, really happy. A few seconds later Fedora said, that it wouldn't boot. The problem was that the included in Fedora stable repository appliance-creator don't adds by default mptspi module to kernel at boot time. The module is required to make LSI Logic controller (default on VMware for SCSI disks) working. Fortunately there was a discussion on that on thincrust-devel list which has produced a patch for appliance-creator. Patched version is still in updates-testing repo. To make an update of appliance-tools (and livecd-tools on which relay they) I run:

yum --enablerepo=updates-testing update livecd-tools appliance-tools

After quick update and appliance recreation. I go a little step forward. Now I got an other error:

mount: error mounting /dev/root on /sysroot as ext3: No such file or directory

This is a well known bug described in Red Hat's Bugzilla #466607 (and in #471093). I found a solution for this problem in comments: add scsi_mod.scan=sync to kernel parameters at boot time. Kickstart file provides a simple way to do it, so I modified our kickstart file in bootloader line:

bootloader --timeout=1 --append="acpi=force scsi_mod.scan=sync"

After another recreation of images, Fedora booted just fine in my ESXi, mission was complete — VMware support was added. I deserve a beer, or not?

I'm going to be in Atlanta for the Atlanta JUG's DevNexus conference, March 10-11.  I may or may not be speaking, but I'll be attending and helping with all things JBoss.

But why should you settle for simply attending the conference, when you could also have me sleep on your sofa? If you live in Atlanta and have a spare sofa or futon, here's your chance to spend some quality time with a JBoss developer.  Invite friends, we'll grill (if you have a grill) and have some beers.

To qualify, you must have a sofa capable of sleeping a 6'5" man.

Leave a comment explaining why I should select your sofa to host the First Atlanta Sofa Conf.  Factors may include who you are, what type of sofa you have, and proximity to the Vinings area, or anything else you deem appropriate.

I just knocked up a quick page on using Maven to build the Java portion of your JBoss-Rails app, if you're doing a hybrid multi-language app.

Anyhow, please check out Using Maven for your mixed-language project.

I'm happy to announce a new version of JBoss Cloud. This is my first release, so I would like to say hi to everyone!

This release contains three new major features (appliance-builder, meta-appliance and VMware ESX support) and a bunch of bug fixes.

Meta-appliance aka metappliance

If you don't want to create your own environment for building appliances on your system, you can use our meta-appliance. We provide it to you in three formats:

• RAW — Ideally to run on KVM,
• VMware Enterprise — Good for VMware ESX/ESXi,
• VMware Personal — Best for VMware Server, Workstation, Fusion, Player etc.

Just grab best format for you from our server and use it. There is a page which describes how to use it.

appliance-builder

Another big change is appliance-builder. It is a wizard for JBoss Cloud appliance building. Just run it from root folder of JBoss Cloud source and see it in action. In next releases there will be more and more features that will make appliance creation quicker and sweeter.

VMware ESX/ESXi support

Beginning with Beta2 JBoss Cloud has support for VMware ESX/ESXi. Yeah, that's true — you can run JBoss Cloud on VMware Infrastructure 3! Just build an VMware enterprise appliance (or grab it from our server!) and push it to VMware ESX/ESXi server via VI Client!

To build an appliance for VMware enterprise product just run:

./appliance-builder -V

Wizard will help you select appliance and best values for it.

Build process will take some time, so go get some beer or coffee. After your'e back, check directory $JBOSS_CLOUD_HOME/build/appliances/$ARCH/$APPLIANCE_NAME/vmware/enterprise/ for appliance.

Unfortunately ESX support requires two packages: livecd-tools-021-1.fc10 and appliance-tools-003.11-1.fc10 that aren't included in fedora stable repositories. New packages are available from updates-testing. Just run the command to update:

yum --enablerepo=updates-testing update livecd-tools appliance-tools

If you don't want to build VMware ESX appliances you can omit the update, of course.

Other stuff

Our yum repository has also a new structure. From Beta2 all appliances could be updated via yum update command!

Notice also that default password for all our appliances is from now oddthesis.

There was also some work on make appliances working in 64 bit environments. 64 bit appliances should work now.

Where can I get it?

Pre-built appliances are available for all supported output formats to download from our server. All appliances are built only for i386 architecture, sorry. If you really want 64 bit support — grab JBoss Cloud sources and build own appliances, that's really easy!

All VMware images are assuming, that you are running those in environment with a network connection named NAT. This is default in VMware personal environments and it should work out of-the-box. However if you want to run it in VMware ESX and NAT doesn't match your configuration (most likely) simply put your network name in nethernet0.networkName parameter in .vmx file.

I you aren't using VM2 for running appliances, you must set manually JBOSS_PROXY_LIST and JBOSS_GOSSIP_HOST in /etc/jboss-as5.conf file in jboss-as5-appliance to have a fully working JBoss AS 5 cluster.

Last words...

Don't forget to take a look at detailed list of tickets done in 1.0.0.Beta2! If you have found a bug or have a good idea how to make JBoss Cloud better — fill a ticket!

If you have any problems, questions on JBoss Cloud bits or just want tell us that you're using it, contact us via IRC or our mailing list. We're happy to hear from you!

In the process of binding a fancy SOAP stack to JBoss-Rails, I of course became frustrated working with DOM trees or StAX iterators.  If you're using 100% Pure Java(tm), you'd use JAXB, and compile up a nice tree matching the object-model described by the XSD in the WSDL.

I, on the other hand, am not using 100% Pure Java(tm) syntax and processes. I'm dynamically deploying WSDLs handled by dynamically reloadable Ruby classes.

So I need a dynamic data-binding to wire up to the Apache-CXF chain.  A CXF DataBinding is responsible for handling readers and writers capable of converting an XML stream into some other object.  And back again.  The data-binding is informed by the XML Schema used to define the web-service.

Now, some examples using the Amazon EC2 WSDL. 

The describe-instances operation is used to list information about running server instances on EC2.  The request message is defined by this XSD fragment:

<xs:element name="DescribeInstances" 
            type="tns:DescribeInstancesType"/>

<xs:complexType name="DescribeInstancesType">
  <xs:sequence>     
    <xs:element name="instancesSet" 
                type="tns:DescribeInstancesInfoType"/>
  </xs:sequence>    
</xs:complexType> 
                
<xs:complexType name="DescribeInstancesInfoType">
  <xs:sequence>     
    <xs:element name="item" 
                type="tns:DescribeInstancesItemType" 
                minOccurs="0" 
                maxOccurs="unbounded"/>
  </xs:sequence>    
</xs:complexType> 
                
<xs:complexType name="DescribeInstancesItemType">
  <xs:sequence>     
    <xs:element name="instanceId" 
                type="xs:string" />
  </xs:sequence>    
</xs:complexType> 

When deployed (at runtime) the Ruby data-binding swizzles that into some relatively simple Ruby classes:

class DescribeInstancesType
  attr_accessor :instancesSet

  def initialize()
    @instancesSet = DescribeInstancesInfoType.new()
  end
end

class DescribeInstancesInfoType < ::Array
end

class DescribeInstancesItemType
  attr_accessor :instanceId

  def initialize()
    @instanceId = ''
  end
end

Through some magic involving subclassing ::Array, when a portion of the tree is array-like, we end up with a very intuitive-to-use model.  We've borrowed this strategy from xsd2ruby by the soap4r guys.

Throwing some XML at it:

<ns1:DescribeInstances>
  <ns1:instancesSet>
    <ns1:item>
      <ns1:instanceId>foo</ns1:instanceId>
    </ns1:item>
    <ns1:item>
      <ns1:instanceId>bar</ns1:instanceId>
    </ns1:item>
    <ns1:item>
      <ns1:instanceId>baz</ns1:instanceId>
    </ns1:item>
  </ns1:instancesSet>
</ns1:DescribeInstances>

Results in the Ruby object:

#<DescribeInstancesType:0x57581ef 
    @instancesSet=[#<DescribeInstancesItemType:0x23356034 
                       @instanceId="foo">, 
                   #<DescribeInstancesItemType:0x242e34f2 
                       @instanceId="bar">, 
                   #<DescribeInstancesItemType:0x6dc22478 
                       @instanceId="baz">]>

And using it is quite simple:

root.instancesSet.each{|e| puts e.instanceId }

Likewise, in a day or two, we'll have the reverse, capable of marshalling a Ruby object right back to some XSD-defined XML.

And all of this will be used in the JBoss-Rails web-services bits.  It's all driven by the Java StAX parsers, so no need to argue about "REXML is slow!" or whatnot.  All XML-handling occurs from Java, against fast Java XML parsers.

I've decided to rework the web-services deployer for JBoss-Rails to try wielding Apache-CXF directly.  The goal is to make configuration even easier.  With JBossWS, it still expected a META-INF/ directory containing your WS-Security configuration file.  In XML.

And we know that's not awesome for Rails folks.

But, during the process of playing with CXF, I found that some of its interceptors are slightly incompatible with the JBoss SOAP DOM implementation.  After spending a day recompiling both CXF and the JBoss DOM bits, I finally located the bug, in CXF's StaxUtils class.

I filed my report in the CXF JIRA, but I don't want to wait around for another release.  I also don't want to mangle the CXF jars I'm getting from the Maven repository or the JBoss DOM classes I get from the app-server.

The quick solution seemed to be replicating the StaxUtils class, with my bugfix, into my own source-tree, and getting it in front of the "official" class located in the CXF jar.

Firing up the JBoss-Rails deployer with my patched class still resulted in failure.  It appears that the jars inside the deployer are loaded alphabetically, perhaps.  CXF vs JBoss, it's hard to get my class in front.

As we've talked previously about structure deployers, and how they help Microcontainer recognize the shape and layout of some thing you deploy.  They figure out where to look for classes, and where to look for meta-data.

MC also supports a declared-structure concept.  The thing-being-deployed tells MC all of this structure information, through a jboss-structure.xml file.  Using this facility within the JBoss-Rails deployer, I have more control over the jar search order:

<?xml version="1.0" encoding="UTF-8"?>
<structure>
    <context>
        <path name=""/>
        <metaDataPath>
            <path name="META-INF"/>
        </metaDataPath>
        <classpath>
            <path name="lib/jboss-rails.jar"/>
            <path name="lib/" suffixes=".jar" />
        </classpath>
    </context>
</structure>

This file says that starting from the top of the deployer's own archive, META-INF/ is for meta-data, and the classpath should include the jboss-rails.jar first, then everything else with a .jar extension.

Once again, Microcontainer saves the day with a handful of metadata.

As time draws nigh, I once again want to remind everyone to go register for the JBoss Virtual Experience.

There are 3 simultaneous tracks: Business, Technical and Goverment.  A lot of JBoss engineers and business people will be around for Q&A after each presentation, plus there's a virtual cafe where folks can linger for informal conversation.

My presentation goes on at 3pm US-Eastern.

It should be fun.