Le carnet à Michel / Mike's Blog / ミシェルのブログ

This is the brand new feature of RHEL5, the ability to have a out of the box virtualization server using a bundled-in version of Xen. However, all is not that easy for the first version. Hence this howto. Note this is subject to changes, these are workarounds for the first version of RHEL5, and may not apply to newer versions.

I need to extend a big thanks to the RHELV5-List users, who carefully helped the newbie I am in the virtualization world.

Goal: To create a Red Hat Enterprise Linux 5 barebones system with virtualization enabled. To create a virtual machine using the same Red Hat Enterprise Linux 5 license.

Lexical note: the host is officially named "Dom0", or domain 0. This is the first operating system you install, and can contain the other VM machines if you want. When you boot your computer, this is what's started. Then, you have VM, or Virtual Machine, that can be started from that Dom0.

1. Burn the different CDs (or DVD), if required, keep the iso images! Insert the first CD onto the machine. Boot using that disk.
2. Create the proper partitions on the disk. This requires a little bit of pondering. You can create partitions as files onto the Dom0, however, you will have slower disk access and you are limited in file size. My solution, using two disks, was to create a Linux boot partition (/boot) on the first disk, the swapfile (swap) on the second disk, and create as many raid partitions as I want to have domains. In my case, I have the Dom0, and I wanted to have 4 VMs, so I created 5 raid partitions on each disk, resulting in the equivalent of 5 disks. Format the first one for your Dom0, the other ones will be used to create your VMs. This was a lengthy process but was well worth it, and it resulted in this (using fdisk -l):
   

   Disk /dev/sda: 250.0 GB, 250000000000 bytes
   255 heads, 63 sectors/track, 30394 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot Start End Blocks Id System
   /dev/sda1 * 1 261 2096451 83 Linux <- This is the /boot disk
   /dev/sda2 262 5330 40716742+ fd Linux raid autodetect <- This is md0 part 1, for Dom0
   /dev/sda3 5331 11596 50331645 fd Linux raid autodetect <- This is md1 part 1, for VM1
   /dev/sda4 11597 30394 150994935 5 Extended <- Created automagically to have more partitions
   /dev/sda5 11597 17862 50331613+ fd Linux raid autodetect <- This is md2 part 1, for VM2
   /dev/sda6 17863 24128 50331613+ fd Linux raid autodetect <- This is md3 part 1, for VM3
   /dev/sda7 24129 30394 50331613+ fd Linux raid autodetect <- This is md4 part 1, for VM4

   Disk /dev/sdb: 250.0 GB, 250000000000 bytes
   255 heads, 63 sectors/track, 30394 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot Start End Blocks Id System
   /dev/sdb1 * 1 261 2096451 82 Linux swap / Solaris <- This is the swapfile, same size as /boot
   /dev/sdb2 262 5330 40716742+ fd Linux raid autodetect <- This is md0 part 2, for Dom0
   /dev/sdb3 5331 11596 50331645 fd Linux raid autodetect <- This is md1 part 2, for VM1
   /dev/sdb4 11597 30394 150994935 5 Extended <- Created automagically to have more partitions
   /dev/sdb5 11597 17862 50331613+ fd Linux raid autodetect <- This is md2 part 2, for VM2
   /dev/sdb6 17863 24128 50331613+ fd Linux raid autodetect <- This is md3 part 2, for VM3
   /dev/sdb7 24129 30394 50331613+ fd Linux raid autodetect <- This is md4 part 2, for VM4

   Disk /dev/md0: 83.3 GB, 83387482112 bytes <- Contains Dom0 LVM
   2 heads, 4 sectors/track, 20358272 cylinders
   Units = cylinders of 8 * 512 = 4096 bytes

   Disk /dev/md0 doesn't contain a valid partition table

   Disk /dev/md4: 103.0 GB, 103078690816 bytes <- Will contain VM4, unformatted for now
   2 heads, 4 sectors/track, 25165696 cylinders
   Units = cylinders of 8 * 512 = 4096 bytes

   Disk /dev/md4 doesn't contain a valid partition table

   Disk /dev/md3: 103.0 GB, 103078690816 bytes <- Will contain VM3, unformatted for now
   2 heads, 4 sectors/track, 25165696 cylinders
   Units = cylinders of 8 * 512 = 4096 bytes

   Disk /dev/md3 doesn't contain a valid partition table

   Disk /dev/md2: 103.0 GB, 103078690816 bytes <- Will contain VM2, unformatted for now
   2 heads, 4 sectors/track, 25165696 cylinders
   Units = cylinders of 8 * 512 = 4096 bytes

   Disk /dev/md2 doesn't contain a valid partition table

   Disk /dev/md1: 103.0 GB, 103078690816 bytes <- Contains VM1
   255 heads, 63 sectors/track, 12531 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot Start End Blocks Id System
   /dev/md1p1 * 1 13 104391 83 Linux
   /dev/md1p2 14 12531 100550835 8e Linux LVM

   From that point on, I created the LVM for Dom0 (as can be seen in fdisk). Here is the pvdisplay:

   — Physical volume —
   PV Name /dev/md0
   VG Name Device0
   PV Size 77.66 GB / not usable 4.50 MB
   Allocatable yes (but full)
   PE Size (KByte) 32768
   Total PE 2485
   Free PE 0
   Allocated PE 2485
   PV UUID —something—

   And a single "main" partition inside the LVM. Why using the LVM? Because it allows resizing afterwards, and is much more flexible. Here is the lvdisplay:

   — Logical volume —
   LV Name /dev/Device0/main
   VG Name Device0
   LV UUID —something—
   LV Write Access read/write
   LV Status available
   # open 1
   LV Size 77.66 GB
   Current LE 2485
   Segments 1
   Allocation inherit
   Read ahead sectors 0
   Block device 253:0

   The main advantage of this technique is the Dom0 system does not know about the other "raid physical drives" and they contain something else from its standpoint. That way, Dom0 cannot mistakenly boot as a Virtual machine, and paths cannot be crossed. Also, I get a notable boost of speed, and I totally separate my 4 VM machines from the Dom0, like if they were different machines altogether.

3. Select what you want to install on Dom0. The only thing you need to install is the Virtualization option.
4. When required, give your installation number, and register to the RHN network, as usual. On first boots, do not forget to update your machine (using yum update, or the shell manager).
5. Install vsftpd (if it wasn't installed). Somehow copy the RHEL5 ISO files onto a folder like your home (~). Name them "diskN.iso", where N is the disk number. Run this script from a terminal:
   

   for i in 1 2 3 4 5; do mount -o loop ~/disc$i.iso /mnt; cp -a /mnt/* /var/ftp/pub; umount /mnt; done

   You can now remove the iso files from the computer, as all its files are copied.

6. Start the FTP server
   

   chkconfig vsftpd on; service vsftpd restart

7. Try to access the files from an external computer, using the IP address of your new machine. If you are unable, you might have to tweak the firewall of your machine to allow FTP access. You might also have to put SELinux in a permissive mode (or deactivate it altogether). This Howto will not show you how to create SELinux policies, that's way beyond the scope of this document. However, it is possible and recommended that at least your Dom0 runs SELinux. That's not for the faint of heart, though. A point here… the SELinux problems I got were all caused with improper vsftpd policies. If you have access to some already installed FTP server on your local network, you should put the files there, that way, no need to install, modify or tweak SELinux. No need to tweak the firewall either. Xen virtualization is totally SELinux-compliant as long as you use partitions (files, you are on your own.)
8. Once you are able to access through external FTP to your machine (ftp://192.168.1.10/pub/ … not ftp://localhost or ftp://127.0.0.1 !), try access a few files, they should properly copy. You should be able to go to Server subfolder and get a few of these files without problem.
9. Now the fun part. Go to the Xen manager, create a new virtual machine. You should put the same amount of memory in both startup and maximum. RHEL5 uses caching system that uses the whole memory block in a matter of minutes anyways. You should use /dev/md1 for your first VM for example. That's even cooler because /dev/md0 is your Dom0, /dev/md1 is your VM1, /dev/md2 is your VM2 and so on
10. When asked if you want paravirtualization, you can say yes, since we want to install a RHEL5 machine inside it, and it's compatible by default.
11. When asked from where to install the disks, they should be available from your IP address you tried before. If you copied the files to /pub, enter ftp://IPADDRESS/pub . Do not point to Server subfolder directly, the system is intelligent enough to determine what to do. There is also provisions for using a http server to install the files. I had problems with that earlier so I cannot really recommend it, however, you might have better luck than me.
12. The VM will boot, it will ask a few questions, it will check for valid media. Now, if you got problems here, it means your FTP server is not properly set up or you do not have the files at the proper place. Make sure all the five CDs are copied on the same folder, and all files are accessible.
13. When asked to connect to RHN network for new subscriptions, just plain forget it. We need to do voodoo later on.
14. Once the VM has installed, it will shut down. To start a VM, you cannot do it from the user interface (!). Go in terminal and do the following line:
    

    xm create /etc/xen/YOUR_VM_NAME

    Yes, this file was miraculously created, and it is named as your server name. The "xm create" command "creates" a VM instance in memory and starts it. Do not ask, misnomer for me, totally inadequate, nigh impossible to find by yourself and not even in the user interface when it's the most important command. But it works. Then you can go back to the GUI and double-click the instance to see its screen and continue installation.

15. Once everything is done, you want to update your machine. For that, you need to register the virtual machine in RHN. Enter terminal and put this line in (With the proper information filled in):
    

    rhnreg_ks --profilename "<myguest>" --username "<myid>" --email "<myemail>" --password "<mypass>" --subscription "<mysubscriptionnumber>"

    It will wait a few seconds, and then return. Once this is done, the ubiquitous "yum update" will work perfectly.

16. One last point. If you want your VM to automatically start with your Dom0, do a "ln" link from /etc/xen/YOUR_VM_NAME to the folder /etc/xen/auto. It will start with your computer.

Enjoy. Send comments, corrections, and horror stories. I made this in 2 hours from past recollection so I cannot vouch everything is perfectly perfect, but it's a real good start.