Configure XenServer 5.0 Free for Software RAID 1
<h3 class="storytitle"><a href="http://www.toddsmith.org/wordpress/server-technology/configure-xenserver-50-free-for-software-raid-1/" rel="bookmark">Configure XenServer 5.0 Free for Software RAID 1</a></h3>http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/<br><br><div class="storycontent">
<p>I used the notes from some guy on the Citrix forums. I wish that I
had his name so I could give him credit but it’s not here on the stuff I
printed out.</p>
<p>On with the show.</p>
<p>To make this work, you need to drives in your system. The second
drive must be identical or larger than the first, and you must have
installed XenServer to the first drive without selecting the second
drive as part of the storage pool.</p>
<p>Install XenServer as usual. Do not select any extra drives as storage pools.</p>
<p>After installation, boot up, and login into console 3 (Alt-F3) as root</p>
<p>type fdisk -l to list the partitions:</p>
<p>Most likely your disks are represented by SCSI device names under linux / XenServer 5.0</p>
<p>in that case your boot disk would be /dev/sda,</p>
<p>To copy the partition table from /dev/sda to /dev/sdb you can use dd</p>
<pre>dd if=/dev/sda of=/dev/sdb bs=512 count=1<br></pre>
<p>Now set the partition table up on /dev/sdb the way it should be for Linux RAID. This means setting the partition types to 0xfd.</p>
<p>I used the following command:</p>
<pre>echo -e "\nt\n1\nfd\nt\n3\nfd\nw\nx" | fdisk /dev/sdb<br></pre>
<p>That tells says to fdisk, “tag partition 1 as type 0xfd, tag partition 3 as type 0xfd”</p>
<p>Check to make sure the /dev/md? devices are present</p>
<pre>[ -e /dev/md0 ] || mknod /dev/md0 b 9 0<br>[ -e /dev/md1 ] || mknod /dev/md1 b 9 1<br></pre>
<p>Startup the degraded RAID devices</p>
<pre>mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1<br>mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb3<br></pre>
<p>The following procedure is directly from the other guys notes. I’ve modified the commands to what I think works a little better.</p>
<pre>pvcreate /dev/md1<br>volume_group=`vgscan | grep VG | awk -F \" '{print $2}'`<br>vgextend $volume_group /dev/md1<br>pvmove /dev/sda3 /dev/md1<br># If this is a fresh install, then there won't be any data to move<br>vgreduce $volume_group /dev/sda3<br></pre>
<p>Now we’re ready to copy the filesystem over to the RAID device /dev/md0</p>
<pre>mkfs.ext3 /dev/md0<br>cd / && mount /dev/md0 /mnt && rsync -a --progress --exclude=/sys --exclude=/proc --exclude=/dev/shm --exclude=/dev/pts / /mnt<br>mkdir /mnt/sys<br>mkdir /mnt/proc<br></pre>
<pre>sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /mnt/etc/fstab<br></pre>
<p>Now let’s setup initrd</p>
<pre>mkdir /root/initrd && cd /root/initrd<br>zcat /boot/initrd-`uname -r`.img | cpio -i && \<br>cp /lib/modules/`uname -r`/kernel/drivers/md/raid1.ko lib<br></pre>
<p>Now we have to edit the init file</p>
<pre>q="echo Waiting for driver initialization."<br>sed -r -i "s,^${q}$,\n\necho Loading raid1.ko module\ninsmod /lib/raid1.ko\n${q}\n,g" init<br>q="resume /var/swap/swap.001"<br>sed -r -i "s,^${q}$,${q}\necho Running raidautorun\nraidautorun /dev/md0\nraidautorun /dev/md1,g" init<br>r=`grep mkroot /root/initrd/init`<br>sed -r -i "s|^${r}$|${r/sda1/md0}|g" init<br></pre>
<p>Now we’ll copy the initial ramdisk to the /boot on the new RAID</p>
<pre>find . -print | cpio -o -c | gzip -c > /boot/initrd-`uname -r`.img<br>sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /mnt/etc/fstab<br>sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /etc/fstab<br></pre>
<p>And setup the boot loader</p>
<pre>sed -r -i 's,root=LABEL=root-\w+ ,root=/dev/md0 ,g' /mnt/boot/extlinux.conf<br>sed -r -i 's,root=LABEL=root-\w+ ,root=/dev/md0 ,g' /boot/extlinux.conf<br>cat /usr/lib/syslinux/mbr.bin > /dev/sdb<br>cd /mnt && extlinux -i boot/<br>extlinux -i boot/<br></pre>
<p>If you’ve done this remotely, you can try the following.</p>
<pre>cp /mnt/boot/extlinux.conf /boot/<br>cp /mnt/boot/initrd-`uname -r`.img /boot<br></pre>
<p>Unmount <code>/dev/md0</code>, <code>sync</code>, and <code>reboot</code></p>
<pre>cd ; umount /mnt || umount /dev/md0<br>sync<br>reboot<br></pre>
<p>First we tag the partitions as type Linux raid, then we have to add <code>/dev/sda</code> to the RAID. </p>
<pre>echo -e "\nt\n1\nfd\nt\n3\nfd\nw\nx" | fdisk /dev/sda<br>mdadm -a /dev/md0 /dev/sda1<br>mdadm -a /dev/md1 /dev/sda3<br></pre>
<h3>Troubleshooting</h3>
<p>The first time I did this procedure I got stuck because my rsync
didn’t copy /proc and /sys like I asked it not to. So I had to add the
step of creating those two directories so they could be mounted at boot.</p>
<p>After messing around with it a little, I came up with this command line to boot XenServer 5 from extlinux.</p>
<pre>mboot.c32 /boot/xen.gz dom0_mem=752M --- /boot/vmlinuz-2.6-xen root=/dev/md0 single --- /boot/initrd-2.6-xen.img<br></pre>
</div>
<div class="feedback">
</div>
<h2 id="comments">22 Comments <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#postcomment" title="Leave a comment">»</a>
</h2>
<ol id="commentlist"><li class="comment even thread-even depth-1" id="comment-82">
<img alt="" src="http://www.gravatar.com/avatar/379073cae7867dab8f162f128cca8694?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>Not booting. Can’t find /dev/root etc.</p>
<p><cite>Comment by <a href="http://www.interdata.lt/" rel="external nofollow" class="url">Ilja</a> — May 26, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-82">6:45 am</a></cite> </p>
</li><li class="comment byuser comment-author-admin bypostauthor odd alt thread-odd thread-alt depth-1" id="comment-84">
<img alt="" src="http://www.gravatar.com/avatar/e624cbc390e6182ce765d35c179c4bfc?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>Did you change your /etc/fstab to point to /dev/md?</p>
<p>Also, you want to make sure to follow the steps to fixup the initrd to ensure that it is loading the correct kernel modules.</p>
<p><cite>Comment by admin — May 26, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-84">7:00 am</a></cite> </p>
</li><li class="comment even thread-even depth-1" id="comment-85">
<img alt="" src="http://www.gravatar.com/avatar/379073cae7867dab8f162f128cca8694?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>Ok, found the spot:</p>
<p>This:<br>
find . -print | cpio -o -c | gzip -c > /boot/initrd-`uname -r`.img</p>
<p>Should be:<br>
find . -print | cpio -o -c | gzip -c > /mnt/boot/initrd-`uname -r`.img</p>
<p>This is done twise in your description. No need:<br>
sed -r -i ’s,LABEL=root-\w+ ,/dev/md0 ,g’ /mnt/etc/fstab</p>
<p>This is also done twise:<br>
extlinux -i boot/</p>
<p><cite>Comment by <a href="http://www.interdata.lt/" rel="external nofollow" class="url">Ilja</a> — May 26, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-85">8:33 am</a></cite> </p>
</li><li class="comment odd alt thread-odd thread-alt depth-1" id="comment-117">
<img alt="" src="http://www.gravatar.com/avatar/05ea70262a5da1fa05c5f5b02995365f?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>Please
let me know if this is working 100% because I wan’t to set this up in 2
production servers. Also, I need Xen to boot automatically from the
other drive in case one of the them is damaged. And mon should send an
email when this happens.</p>
<p>Please let me know the correct steps on this.</p>
<p>Keep up the good work <img src="http://www.toddsmith.org/wordpress/wp-includes/images/smilies/icon_smile.gif" alt=":)" class="wp-smiley"> </p>
<p><cite>Comment by organetic — June 10, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-117">5:41 am</a></cite> </p>
</li><li class="comment byuser comment-author-admin bypostauthor even thread-even depth-1" id="comment-119">
<img alt="" src="http://www.gravatar.com/avatar/e624cbc390e6182ce765d35c179c4bfc?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>It’s
working for me in production environments. It’s Linux software RAID
which is very good RAID code. Also this configuration boots off of the
RAID device. So as long as at least one of your disks are good, the
system should boot.</p>
<p><cite>Comment by admin — June 10, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-119">6:48 am</a></cite> </p>
</li><li class="comment odd alt thread-odd thread-alt depth-1" id="comment-162">
<img alt="" src="http://www.gravatar.com/avatar/8c228775622e489d5e64b392a94b9d5a?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>After
following your tutorial including the adjustment pointed out by Ilja, I
have a successful install of Xen w/ software RAID 1. At this point I am
trying to re-build the array in another (idnetical box). When
connecting using XenCenter I get an error saying that it cannot connect
to both Xen boxes because one is a backup of the other. Do you know what
I can change so the boxes can both be connected to simultaneously?</p>
<p><cite>Comment by <a href="http://thewiredmouse.com/" rel="external nofollow" class="url">tdb</a> — June 22, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-162">11:11 am</a></cite> </p>
</li><li class="comment byuser comment-author-admin bypostauthor even thread-even depth-1" id="comment-169">
<img alt="" src="http://www.gravatar.com/avatar/e624cbc390e6182ce765d35c179c4bfc?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>Andrew,</p>
<p> I’m sorry I don’t know what needs to be changed. I’m sure it’s some
identifier that is unique to each installation. If you find out, please
let me know.</p>
<p>-Todd</p>
<p><cite>Comment by admin — June 23, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-169">9:20 pm</a></cite> </p>
</li><li class="comment odd alt thread-odd thread-alt depth-1" id="comment-194">
<img alt="" src="http://www.gravatar.com/avatar/379073cae7867dab8f162f128cca8694?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>tdb,
it could be that extlinux is written only on the first drive. When raid
is built, you need to write “boot loader code” to the second HDD.
Unfortunately I don’t know how to do it with extlinux, haven’t work with
it much.</p>
<p>With 2 HDD configuration using grub it would be:<br>
# grub<br>
grub> device (hd1) /dev/sdb<br>
device (hd1) /dev/sdb<br>
grub> root (hd1,0)<br>
root (hd1,0)<br>
Filesystem type is ext2fs, partition type 0xfd<br>
grub> setup (hd1)<br>
setup (hd1)<br>
Checking if “/boot/grub/stage1″ exists… no<br>
Checking if “/grub/stage1″ exists… yes<br>
Checking if “/grub/stage2″ exists… yes<br>
Checking if “/grub/e2fs_stage1_5″ exists… yes<br>
Running “embed /grub/e2fs_stage1_5 (hd1)”… 15 sectors are embedded.<br>
succeeded<br>
Running “install /grub/stage1 (hd1) (hd1)1+15 p (hd1,0)/grub/stage2 /grub/grub.conf”… succeeded<br>
Done.<br>
grub> quit<br>
#</p>
<p>Good luck.</p>
<p>- Ilja</p>
<p><cite>Comment by <a href="http://www.interdata.lt/" rel="external nofollow" class="url">Ilja</a> — June 29, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-194">12:23 pm</a></cite> </p>
</li><li class="comment even thread-even depth-1" id="comment-288">
<img alt="" src="http://www.gravatar.com/avatar/ebff23095e2bb41dd55648aab49cea5f?s=32&d=http%3A%2F%2Fwww.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D32&r=G" class="avatar avatar-32 photo" width="32" height="32"> <p>Hi. Can you advise whether this works for Free XenServer 5.5</p>
<p><cite>Comment by David — July 18, 2009 @ <a href="http://www.toddsmith.org/wordpress/uncategorized/configure-xenserver-50-free-for-software-raid-1/#comment-288">8:06 am</a></cite> </p>
</li></ol><br><br>