Marek Goldmann | 03:08pm UTC, 23 February 2009
I'm a VMware fan, really. They provide good, reliable virtualization environment for personal use and big business too. In the middle of 2008 VMware has announced a new product called ESXi. It's a hypervizor, available for free (but still not opensource), for everyone. ESXi can be a part of VMware Infrastructure (you can choose between old, fat ESX Server and new, light ESXi).
Unfortunately ESXi can be installed only on certified hardware. Officially. Many people around the world have created a white-box list of systems (mainboards, controllers, etc.) capable running ESXi, so you can pick up a good, cheap piece of hardware and have really good virtualization at home (or company).
Before I started digging in JBoss-Cloud I had a wish to run it on ESXi (and VMware Infrastructure 3 too as formats are the same), so making JBoss-Cloud on ESXi working was my priority (VMware personal support was already done by Bob).
VMware disk types
For appliance image creation we use appliance-creator tool from Thincrust. Te default output format of this tool is a RAW disk. It's great for KVM, but not so good for VMware, or rather I thought so before... But, in turn.
VMware uses seven (sic!) disk types. We can exclude types which are breaking a disk in 2 GB files. We don't need them as we are using good filesystems, right? We don't need also physical disks. After excludes, we have still three types; two for VMware Personal: monolithicSparse, monolithicFlat and one for VMware Enterprise: vmfs.
BTW, I'm using the unofficial VMware Personal and Enterprise terms to distinguish VMware products intended to personal, home use (Server, Fusion, Workstation, Player) and enterprise, datacenter use (Infrastructure, ESXi).
So, back to disks. What's the main difference between monolithicSparse and monolithicFlat? It's simple: monolithicSparse is growing as there is more data to store on the disk and monolithicFlat allocates all disk size at the time of creation. Another difference is that monolithicSparse has a embedded disk descriptor, monolithicFlat has an external descriptor file.
Thincrust appliance-creator has support for various disk formats. How it is done? It creates a RAW disk image and, if needed, it converts it to a specified format using qemu-img. As qemu-img has a wide support for various formats, appliance-creator is a very handy tool, worth taking a look (or two)!
VMware disk support is also included in qemu-img. Unfortunately only monolithicSparse disk type is supported by this tool and we need vmfs. Second glitch: created disk is a IDE disk. It's fine for VMware Personal, but VMware Enterprise uses only SCSI disks. As the disk descriptor for monolithicSparse is embedded, there is no simple way to convert it to a SCSI disk (at least I don't know any simple method).
We need to boot it, now!
I understood that if I want to have JBoss-Cloud working on VMware Enterprise I must do it another way. After some time spent on vmfs disk format I realized that this is... simply a RAW disk! So, really good news, as we have a RAW disk already!
In VMware environment a disk needs also a disk descriptor file. For vmfs disk type the descriptor is fortunately an external, simple text file which can be edited by hand. I've created one for my disk with all needed information (disk size, geometry, SCSI controller etc.). It wouldn't be possible without a great site from Ulli Hankeln. Ulli has described the vmfs descriptor format and created also a disk geometry table for every disk size in the world. Thank you Ulli, great job!
After first run I saw the grub menu. I was happy, really happy. A few seconds later Fedora said, that it wouldn't boot. The problem was that the included in Fedora stable repository appliance-creator don't adds by default mptspi module to kernel at boot time. The module is required to make LSI Logic controller (default on VMware for SCSI disks) working. Fortunately there was a discussion on that on thincrust-devel list which has produced a patch for appliance-creator. Patched version is still in updates-testing repo. To make an update of appliance-tools (and livecd-tools on which relay they) I run:
yum --enablerepo=updates-testing update livecd-tools appliance-tools
After quick update and appliance recreation. I go a little step forward. Now I got an other error:
mount: error mounting /dev/root on /sysroot as ext3: No such file or directory
This is a well known bug described in Red Hat's Bugzilla #466607 (and in #471093). I found a solution for this problem in comments: add scsi_mod.scan=sync to kernel parameters at boot time. Kickstart file provides a simple way to do it, so I modified our kickstart file in bootloader line:
bootloader --timeout=1 --append="acpi=force scsi_mod.scan=sync"
After another recreation of images, Fedora booted just fine in my ESXi, mission was complete — VMware support was added. I deserve a beer, or not?