Yesterday we basically reinstalled the main host for the cduce and mancoosi projects. The Problem was that the machine (a power edge 2950) was installed with a 32 bits system while the Xeon processors are 64 bits. To cut the story short we decided to re-install the system.
First we installed a generic 64 bits kernel. Debian ships this kernel in the i386 repository, so it was as simple as apt-get install. After we reboot the machine, we had to add a new lvm partition for the new 64 installation and debootstrap a new system in it.
The new system worked straightaway, so we did a bit of data shuffling and we end up with a working 64 bits system.
Now the hard part. We run 4 guest os on this machine. These guest Oses were all 32 bits system and we find out the hard way that is not possible to run a 32 bit Os on a 64 bit system without using a new Xen (3.3) and the HVM emulation layer.
After a few desperate experiments we decided to re-install the guest Oses as 64 bits system. After xen-create all these images and a bit more of lvm shuffling and renaming, we were ready to re-configure our machines.
Using the old dpkg —get-selections trick we were able pretty quickly to re-install the needed software. Of course I had to fight a bit with apt-get to manually solve a few dependencies problems.
Now the big mistake. I thought that the vserver image had to be installed on each guest machine. This is wrong ! Installing the vserver image is not necessary and actually this move gave me quite an headache as it pulled in also udev.
Never, ever install udev on a xen emulated os ! Well, after a bit of googling, I got the right brain wave, I di-installed udev and all machines booted just fine.
Ldap: the problem is that is the to copy an ldap database, is necessary to dump the db in a text file and the upload it again in a new fresh db. Needless to say I forgot to back the ldap db. So to save the day, I installed ldap on my (32 bits) machine, dump the db and then with a old good ldif file in my hand I was able to re start ldap without loosing anything.
Mysql was a bit the same problem. It seems that the InnoDB tables don’t like to be moved around. In order to get mysql running again I had to dump the bd, remove everything and upload the data back again. This was less painful, but still a bit time consuming.
the last hurdle was about uids. Re-installing the system I didn’t realized that numeric ids are not fixed, but given sequentially (well, of course I knew it ! just didn’t think about this problem …) so the uids on the old images were different from the uids on the new images. It took only a few minutes and a find script to fix the problem.
Now everything seems up and running. We see 8Gb of memory as we should and we apart for other minor quirks that I’m sure will come up soon, I’m quite happy about the move. I said it was a 30 mins job, it took 8 hs, but still pretty good time !