After the upgrade of last week, I didn’t have any major problems : xen 4 seems pretty stable and does its job well. One problem I encountered the other day was about the dom0 balloning. By default, xen sets dom0_min_mem to 196Mb and balloning set to true. This is all and good untill you try to use too much memory for your VMs, squeezing dom0 to its minimum amount of memory and causing all sort of problems. On the xen wiki, they reccomend as best practice to reserve a minimum of 512Mb to dom0 for its operations. This is done but setting dom0_mem=512M on the grub command line and the adjusting enable-dom0-ballooning to no and dom0-min-mem accordingly to the amount of memory you choose.

On debian, you can set the grub command line once for all just by adding in /etc/default/group the conf variable

GRUB_CMDLINE_XEN="dom0_mem=512M"

Another small problem is related to the reboot sequence. Since I’m using lvm on aoe, the default shutdown sequence (network down first, lvm later) is not going to work for me. As I’ve few lvm volumes on aoe and others on the physical disk, the proper solution to this problem is to write a custom shutdown script for the aoe lvm volumes and make it run before deconfiguring the network interfaces. In the mean time, to avoid the kernel hanging there forever, I’ve added these lines to in /etc/syscntl.d/panic.conf

# Reboot 5 seconds after panic
kernel.panic = 5

# Panic if a hung task was found
kernel.hung_task_panic = 1

# Setup timeout for hung task to 300 seconds
kernel.hung_task_timeout_secs = 120

This will instruct the kernel to panic and then reboot if there a task will not respond for more then 120 seconds.

It’s time to upgrade my xen servers to squeeze. I’ve already put this off too long and now I’ve to task to go from etch to squeeze in one long step. In order to avoid problems I just did a first upgrade etch -> lenny and then to squeeze. However, since so much has changed in the meantime, and so much twicking of essential components is needed (such as Xen !), I guess I could have gone directly from etch to squeeze in one go, and fix everthing in the process… Anyway, to late for this kind of considerations.

The xen debian wiki is full of invaluable information. Kudos to the xen team for their hard work. To get you started on squeeze you need to install the xen hypervisor. Everything is provided by one package:

aptitude install xen-linux-system-2.6-xen-amd64 xen-hypervisor-4.0-amd64

This will pull the latest linux kernel and xen-hypervisor to run on dom0 .

By default the hypervisor is probably not going to be the default kernel. If you want to change this, you should edit the grub default values :

vi /etc/default/grub

to make sure that the default kernel on dom0 is the xen-hypervisor. This is tricky, because grub let you define a default w.r.t the list of available kernels. so if you install a new kernel, you have to change the default accordingly with the list of kernels in /boot/grub/grub.cfg. It would be nice if I could define the default kernel with a label instead of a number… ( ref #505517 )

Alternatively, as suggested in the wiki, you can just move the xen kernel out of the way …

mv -i /etc/grub.d/10_linux /etc/grub.d/50_linux

When installing xen related tools, aptitude will also probably install by default rinse and xenwatch. The first one is to boostrap redhat machines and maybe you don’t need it. The second one is a GUI and will pull in a lot of X-related dependencies. If we have similar needs, you can just remove what is not needed…

aptitude purge rinse rpm rpm-common
aptitude purge xenwatch

Something that is new, is the new schema for virtual devices. Now all vms will see /dev/xdva1 instead of /dev/sda1 as before. This needs to be changed in the domU as well as in the xen config files (/etc/xen/vm.cfg).

One fantastic news is that xen 4 now uses pyGrub. It is not mandatory (so if you want, you can stick with the old configuration file). But if you use pygrub, on the domU you can install whatever kernel you want. Finally, your users will have complete freedom to pick and choose their kernels !

There was a small detail I didn’t notice on the debian wiki, that is, if you try to use grub2 in squeeze, it will fail when probing the device#601974 . The workaround described in the wiki is to use xvd{a,b,c,…} as device names (and not xvd{1,2,3,…}) to make grub happy. Once you have changed then naming schema, grub will be able to see the disks and install the bootloader. Another solution is to install the os-prober from unstable / experimental. It seems a patch is on the works.

On newly created images, you can also pass the —scsi parameter to xen-create-image to ignore this problem altogeher… I’m not sure if this will have other implications…

The console name is also changes from tty to xhv0 . To get back the console you should add this line in the inittab of all you VMs.

vc:2345:respawn:/sbin/getty 38400 hvc0

A last note is about the [http://blog.xen.org/index.php/2011/01/14/linux-2-6-37-first-upstream-linux-kernel-to-work-as-dom0/ merge upstream of the xen patch !!] \o/ yeiiii !!

The Mancoosi Team has recently published the details of a study we conducted analyzing different packages managers available in debian. The goal of this study was to compare MPM (the mancoosi package manager) to other legacy solvers and try to get a big picture regarding of the state of the art. A similar study was conduct during EDOS and the results are still available here.

As I wrote few months ago, MPM is a proof of concept that we wrote to test the behavior of a number of solvers that have been developed for the MISC competition in a real world scenario.

These results do not show anything new w.r.t. the experience of a lot of poeple dueling daily with their machines in order to install a new piece of software. In a nutshell, we have shown that apt-get, aptitude, smart and cupt perform pretty well when used only with one baseline (for example a stable release) : this conforms with the experience of the majority of users of debian based systems. However problems start to arise when a user start mixing more then one baseline putting a lot of stress on the solver in order to find a satisfying solution. This solution of course exists, but it is cleverly hidden in the dependency structure of more then 40K packages…

MPM is not as fast as other package solvers (say 10 seconds for mpm, while apt-get is able to find a solution in 3 second), but is remarkably stable. It is always able to find a satisfying solution, even in the harder cases where all the other failed. In these experiments MPM uses the postdam solver aspcud. This solver uses only GPL components and it would be a good candidate for inclusion in debian (there are actually a couple of ITP already filled for clasp and gringo).

The results (with a lot of details) are published on the mancoosi website (a more detailed report is in the works). Enjoy !

During fosdem the Mancoosi team that authored this work (Roberto, Ralf, Zack and Me) will be around, so, please stop us for a chat ! And don’t miss Ralf’s talk !!

And don’t miss the talk from Ralf at the crossdistro devroom [1] …

Sat 05/02 18:00 - 19:00: Mancoosi tools for the analysis and quality assurance of FOSS distributions (Ralf Treinen)

[1]http://fosdem.org/2011/preview-saturday#crossdistro_devroom

Update

There is an easier method to do all this using gbp-clone as described here. Ah !

Then to build the package, you just need to suggest git-buildpackage where to find the pristin-tar :

git-buildpackage --git-upstream-branch=upstream/master

or you could simply describe (as suggested) the layout in debian/gbp.conf.

Easy !!!

I’ve found a lot of different recipes and howtos about git debian packaging, but I failed to find one simple recipe to create a debian package from scratch when upstream is using git. Of course the following is a big patchwork from many different sources.

First we need to do a bit of administrative work to setup the repository :

mkdir yourpackage
cd yourpackage
git init --shared

Then, since I’m interested in tracking upstream development branch I’m going to add a remote branch to my repo:

git remote add upstream git://the.url/here.git

at this point I need to fetch upstream and create a branch for it.

git fetch upstream
git checkout -b upstream upstream/master

Now in my repo I have a master branch and an upstream branch. So far, so good. Let’s add the debian branch based on master:

git checkout master
git checkout -b debian master

It’s in the debian branch where I’m going to keep the debian related files. I’m finally read for hacking git add / git commit / git remove ...

When I’m done, I can switch to master, merge the debian branch into it and use git-buildpackage to build the package.

git checkout master
git branch 
  debian
* master
  upstream

git merge debian
git-buildpackage

Suppose I want to put everything on gitourious for example. I’ll create an acocunt, set up my ssh pub key and then I’ve to add an origin ref in my .git/config . Something like :

[remote "origin"]
       url = git@gitorious.org:debian-stuff/yourpackage.git
       fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
       remote = origin
       merge = refs/heads/master

The only thing left to do is to push everything on gitourious. the —all is important.

git push --all

People willing to pull your work from girourious have to follow the following script :

$git clone git@gitorious.org:debian-stuff/yourpackage.git
$cd yourpackage
$git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/debian
  remotes/origin/master
  remotes/origin/upstream
$git checkout -t origin/debian
$git checkout -t origin/upstream
$git branch -a
  debian
  master
* upstream
  remotes/origin/HEAD -> origin/master
  remotes/origin/debian
  remotes/origin/master
  remotes/origin/upstream
$git checkout master
$git-buildpackage

Maybe there is an easier way to pull all remote branches at once, but I’m not aware of it. Any better way ?