I’ll start here a small series of posts about ganeti, xen and puppet. For my work I run few servers sitting on xen and it has always been a bit of a pain to create a new instance and keep it up to date. Up to now I’ve used the excellent xen-create-image tool to create my VMs, but I wanted to try something new and more sexy… Last week I finally found some time to learn (and a spare box to run my experiments) how to use ganeti. Ganeti is the only tool I tried out, but it seems to fit the bill for my use and it seems polished and mature project to me… Moreover I’ve seen a presentation about it in every FLOSS conference I’ve attended in the last few years and I thought it was time to give it a try.

Installing and configuring ganeti is fairly easy, there is a lot of documentation available and this post is not going to be about installing it, but rather how to create a new bare instance with ganeti-deboostrap-instance. There is also a way to create a new instance from an image, but I didn’t go that way yet.

This first post is about the first problem I’ve encountered, that is, how to automatically assign a network address and a name to each new instance created by gnt-instance add. Since all my instances should be able to communicate together on the same subnet, I’ve decided to configure xen to create a NATted private network and add each new instance to this network.

The first step is to create an interface in /etc/network/interfaces .

auto xen-br0
iface xen-br0 inet static
    address 10.0.0.1
    netmask 255.255.255.0
    bridge_stp off
    bridge_fd 0
    bridge_ports none

This is the standard debian way but since xen uses a different naming convention (here I’m using ganeti naming convention xenbr0 vs xen-br0), I need to convince tell xen what I intend to do by adding these lines in /etc/xen/xend.config :

(network-script 'network-virtual bridgeip="10.0.0.1/24" brnet="10.0.0.0/24" bridge="xen-br0"')
(vif-script     vif-bridge)

Next I have to connect my real network interface to the private network using few iptables rules in /etc/rc.local (probably there is a better place to do this…):

echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
/sbin/iptables -A FORWARD -i eth0 -o xen-br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
/sbin/iptables -A FORWARD -i xen-br0 -o eth0 -j ACCEPT

The xen setup is complete and every new image should have a vif connected to the subbet 10.0.0.0. The xen setup corresponds to the physical wiring of the network. The next step is to configure each instance so to allow them to communicate on this subnet. Since I build my VMs using ganeti-debootstrap-instance, and by default debootstrap does not configure the network, we need to add a new hook in the directory /etc/ganeti/instance-debootstrap/hooks.

#!/bin/bash
if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
  echo "Missing target directory"
  exit 1
fi

if [ ! -d "$TARGET/etc/network" ]; then
  echo "Missing target network directory"
  exit 1
fi

if [ -z "$NIC_COUNT" ]; then
  echo "Missing NIC COUNT"
  exit 1
fi

if [ "$NIC_COUNT" -gt 0 ]; then

  cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

EOF

fi

DAEMON_PID_FILES="/var/run/dnsmasq.pid /var/run/dnsmasq/dnsmasq.pid"

instance=$INSTANCE_NAME
[ -n "$instance" ] || exit 1
nic_count=$((NIC_COUNT - 1))
mac_var="NIC_${nic_count}_MAC"
echo $mac_var
echo $nic_count
mac=${!mac_var}
echo $mac
echo "dhcp-host=$mac,$instance" > /etc/dnsmasq.d/$instance.conf

This hook will do two things. First it will configure the interfaces of the new instance to get configured using dhcp, second, it will add an entry to the dnsmasq configuration file to make this instance known to the world. This basically boils down to add a file in /etc/dnsmasq.d/ with the mac address of the new instance and its designated name. Dnsmasq will then provide an ip address for this instance and add it to the dns.

dhcp-host=aa:00:00:24:6c:8a,node1

Configuring dnsmasq is pretty easy as well. First I want it to answer dhcp queries only on the internal network, second I want to configure my clients passing 10.0.0.1 as nameserver and gataway. You can just add the following lines in /etc/dnsmasq.d/general to get it going.

interface=xen-br0
interface=lo
dhcp-range=10.0.0.128,10.0.0.250
domain=localnet.org,10.0.0.128,10.0.0.250
dhcp-option=3,10.0.0.1
bogus-priv
#expand-hosts
local=/localnet.org/

To create your new instance you can just run the following command :

gnt-instance add -t plain -s 5g -B memory=1024 -o  debootstrap+unstable --no-ip-check --no-name-check node1

If you are running your dom0 on debian squeeze before running this command you should configure ganeti to pass the right xen parameters to the newly created instance :

gnt-cluster modify --hypervisor-parameter xen-pvm:root_path='/dev/xvda1'
gnt-cluster modify --hypervisor-parameter xen-pvm:initrd_path='/boot/initrd-2.6-xenU'

I use —no-ip-check and —no-name-check to skip ip and dns check performed by ganeti and to avoid a sort of chicken-egg problem, where the name and address of this new instance is yet unknown to dnsmasq and that node1 is the name that will be used by the hook to add an entry in the dnsmasq configuration. debootstrap+unstable is a variant of the default configuration and you need to add it to the list of variants used by ganeti-deboostrap-instance.

That should be it. The new instance should come up with a dynamically assigned ip address, able to talk to the outside world and automatically known by all the other machine on the subnet via dns.

Next post will be about how to add a swap hook for ganete-debootstrap-istance.

I’m blogging about this small configuration issue as it took me some time to figure out how to configure cupt and smart to solve this problem. The reason I’m playing with cupt and smartpm is that I’m working to compare again a number of package managers in debian against the state of the art cudf solvers using mpm, and I’m suffering quite a bit to configure my virtual environment. Last year I promised to revise and fix our results. I didn’t forget my promise, but it seems it took longer then expected.

Anyway, back to the main topic. The release-problem arises because the key used to sign sarge (that I’m using as baseline for my experiments) is long expired. If you try to retrieve sarge from archive.debian.org you will find that sarge is signed with the key A70DAF536070D3A1 and apt-get will complain loudly if you try to use an archive signed with an expired key.

cupt

For cupt this is documented in the man page and there are a number of options to add either to /etc/apt/apt.conf to the the cupt own conf file. Then cupt will happily accept the sarge Packages file and let you run update.

cupt::cache::release-file-expiration::ignore "true";
cupt::update::check-release-files "false";
cupt::update::keep-bad-signatures "true";

smartpm

For smart I could not found this information anywhere, but reading the source code (tnx good it’s python !). To cut it short, you need to set the key trustdb to an empty value for a specific channel. On the command line you get something like :

smart channel --set aptsync-614482cb2c7e08d5722af3498232ba52 keyring= --config-file=/root/var/lib/smart/config

where aptsync-614482cb2c7e08d5722af3498232ba52 is the channel name corresponding to sarge in my conf. Since I’m using a simulated environment, I save the result of this option in a non-default config file in my chroot.

Today I installed unburden-home-dir and I’m very please with it. It’s a simple script that takes care of your temporary files to move them outside your home directory. The main reason why I installed it is to minimize the number of read/writes of iceweasel. My favorite browser is apparently the culprit of 80% of read / write operations on disk even when it is idle… Moving the cache to tmpfs, I hope to reduce the IO on disk and to extend my battery life. Using a SSD I haven’t noticed any remarkable benefits regarding performances, but I hope I’ll manage to squeeze a bit more from my battery.

Installing and configuring unburden-home-dir is straightforward. It is packaged for debian (experimental at the time of writing), and it is very easy to configure. Remember that if you want to have your cache on tmpfs, you need to ether mount a tmpfs-enabled file system somewhere or enable RAMTMP=yes in /etc/default/rcS (default in wheezy).

After installing unburden-home-dir iotop shows a delightful page full of zeros :)

ps: if you use duplicity, remember to specify the option —archive-dir to move the duplicity cache somewhere else…