I used the notes from some guy on the Citrix forums. I wish that I had his name so I could give him credit but it’s not here on the stuff I printed out.
On with the show.
To make this work, you need to drives in your system. The second drive must be identical or larger than the first, and you must have installed XenServer to the first drive without selecting the second drive as part of the storage pool.
Install XenServer as usual. Do not select any extra drives as storage pools.
After installation, boot up, and login into console 3 (Alt-F3) as root
type fdisk -l to list the partitions:
Most likely your disks are represented by SCSI device names under linux / XenServer 5.0
in that case your boot disk would be /dev/sda,
To copy the partition table from /dev/sda to /dev/sdb you can use dd
dd if=/dev/sda of=/dev/sdb bs=512 count=1
Now set the partition table up on /dev/sdb the way it should be for Linux RAID. This means setting the partition types to 0xfd.
I used the following command:
echo -e "\nt\n1\nfd\nt\n3\nfd\nw\nx" | fdisk /dev/sdb
That tells says to fdisk, “tag partition 1 as type 0xfd, tag partition 3 as type 0xfd”
Check to make sure the /dev/md? devices are present
[ -e /dev/md0 ] || mknod /dev/md0 b 9 0
[ -e /dev/md1 ] || mknod /dev/md1 b 9 1
Startup the degraded RAID devices
mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb3
The following procedure is directly from the other guys notes. I’ve modified the commands to what I think works a little better.
pvcreate /dev/md1
volume_group=`vgscan | grep VG | awk -F \" '{print $2}'`
vgextend $volume_group /dev/md1
pvmove /dev/sda3 /dev/md1
# If this is a fresh install, then there won't be any data to move
vgreduce $volume_group /dev/sda3
Now we’re ready to copy the filesystem over to the RAID device /dev/md0
mkfs.ext3 /dev/md0
cd / && mount /dev/md0 /mnt && rsync -a --progress --exclude=/sys --exclude=/proc --exclude=/dev/shm --exclude=/dev/pts / /mnt
mkdir /mnt/sys
mkdir /mnt/proc
sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /mnt/etc/fstab
Now let’s setup initrd
mkdir /root/initrd && cd /root/initrd
zcat /boot/initrd-`uname -r`.img | cpio -i && \
cp /lib/modules/`uname -r`/kernel/drivers/md/raid1.ko lib
Now we have to edit the init file
q="echo Waiting for driver initialization."
sed -r -i "s,^${q}$,\n\necho Loading raid1.ko module\ninsmod /lib/raid1.ko\n${q}\n,g" init
q="resume /var/swap/swap.001"
sed -r -i "s,^${q}$,${q}\necho Running raidautorun\nraidautorun /dev/md0\nraidautorun /dev/md1,g" init
r=`grep mkroot /root/initrd/init`
sed -r -i "s|^${r}$|${r/sda1/md0}|g" init
Now we’ll copy the initial ramdisk to the /boot on the new RAID
find . -print | cpio -o -c | gzip -c > /boot/initrd-`uname -r`.img
sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /mnt/etc/fstab
sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /etc/fstab
And setup the boot loader
sed -r -i 's,root=LABEL=root-\w+ ,root=/dev/md0 ,g' /mnt/boot/extlinux.conf
sed -r -i 's,root=LABEL=root-\w+ ,root=/dev/md0 ,g' /boot/extlinux.conf
cat /usr/lib/syslinux/mbr.bin > /dev/sdb
cd /mnt && extlinux -i boot/
extlinux -i boot/
If you’ve done this remotely, you can try the following.
cp /mnt/boot/extlinux.conf /boot/
cp /mnt/boot/initrd-`uname -r`.img /boot
Unmount /dev/md0
, sync
, and reboot
cd ; umount /mnt || umount /dev/md0
sync
reboot
First we tag the partitions as type Linux raid, then we have to add /dev/sda
to the RAID.
echo -e "\nt\n1\nfd\nt\n3\nfd\nw\nx" | fdisk /dev/sda
mdadm -a /dev/md0 /dev/sda1
mdadm -a /dev/md1 /dev/sda3
Troubleshooting
The first time I did this procedure I got stuck because my rsync didn’t copy /proc and /sys like I asked it not to. So I had to add the step of creating those two directories so they could be mounted at boot.
After messing around with it a little, I came up with this command line to boot XenServer 5 from extlinux.
mboot.c32 /boot/xen.gz dom0_mem=752M --- /boot/vmlinuz-2.6-xen root=/dev/md0 single --- /boot/initrd-2.6-xen.img
Not booting. Can’t find /dev/root etc.
Comment by Ilja — May 26, 2009 @ 6:45 am
Did you change your /etc/fstab to point to /dev/md?
Also, you want to make sure to follow the steps to fixup the initrd to ensure that it is loading the correct kernel modules.
Comment by admin — May 26, 2009 @ 7:00 am
Ok, found the spot:
This:
find . -print | cpio -o -c | gzip -c > /boot/initrd-`uname -r`.img
Should be:
find . -print | cpio -o -c | gzip -c > /mnt/boot/initrd-`uname -r`.img
This is done twise in your description. No need:
sed -r -i ’s,LABEL=root-\w+ ,/dev/md0 ,g’ /mnt/etc/fstab
This is also done twise:
extlinux -i boot/
Comment by Ilja — May 26, 2009 @ 8:33 am
Please let me know if this is working 100% because I wan’t to set this up in 2 production servers. Also, I need Xen to boot automatically from the other drive in case one of the them is damaged. And mon should send an email when this happens.
Please let me know the correct steps on this.
Keep up the good work " class="wp-smiley">
Comment by organetic — June 10, 2009 @ 5:41 am
It’s working for me in production environments. It’s Linux software RAID which is very good RAID code. Also this configuration boots off of the RAID device. So as long as at least one of your disks are good, the system should boot.
Comment by admin — June 10, 2009 @ 6:48 am
After following your tutorial including the adjustment pointed out by Ilja, I have a successful install of Xen w/ software RAID 1. At this point I am trying to re-build the array in another (idnetical box). When connecting using XenCenter I get an error saying that it cannot connect to both Xen boxes because one is a backup of the other. Do you know what I can change so the boxes can both be connected to simultaneously?
Comment by tdb — June 22, 2009 @ 11:11 am
Andrew,
I’m sorry I don’t know what needs to be changed. I’m sure it’s some identifier that is unique to each installation. If you find out, please let me know.
-Todd
Comment by admin — June 23, 2009 @ 9:20 pm
tdb, it could be that extlinux is written only on the first drive. When raid is built, you need to write “boot loader code” to the second HDD. Unfortunately I don’t know how to do it with extlinux, haven’t work with it much.
With 2 HDD configuration using grub it would be:
# grub
grub> device (hd1) /dev/sdb
device (hd1) /dev/sdb
grub> root (hd1,0)
root (hd1,0)
Filesystem type is ext2fs, partition type 0xfd
grub> setup (hd1)
setup (hd1)
Checking if “/boot/grub/stage1″ exists… no
Checking if “/grub/stage1″ exists… yes
Checking if “/grub/stage2″ exists… yes
Checking if “/grub/e2fs_stage1_5″ exists… yes
Running “embed /grub/e2fs_stage1_5 (hd1)”… 15 sectors are embedded.
succeeded
Running “install /grub/stage1 (hd1) (hd1)1+15 p (hd1,0)/grub/stage2 /grub/grub.conf”… succeeded
Done.
grub> quit
#
Good luck.
- Ilja
Comment by Ilja — June 29, 2009 @ 12:23 pm
Hi. Can you advise whether this works for Free XenServer 5.5
Comment by David — July 18, 2009 @ 8:06 am