I’ve recently added a hook in my git repository to send all my commits to identi.ca. There are a plethora of methods to do so, and I’ve chosen bti, that is a small program, available in debian with oauth support.

Following the instructions I found here :

• register an application at https://identi.ca/settings/oauthapps/new
• add consumer_key and consumer_secret into ~/.bti_oauth
• run bti:

$ bti --config ~/.bti_oauth
Please open the following link in your browser, and allow 'bti' to access your account. Then paste back the provided PIN in here.
http://identi.ca/api/oauth/authorize?oauth_token=$token
PIN: $pin_copied_from_above_URL
Please put these two lines in your bti configuration file (~/.bti):
access_token_key=$key
access_token_secret=$secret

• add access_token_key and access_token_secret to ~/.bti_oauth
• try to post something:

$ bti --config ~/.bti_oauth
tweet: testing bti with oauth on identi.ca ...

Once you have bti working, you can add the following script in your post-receive script in the directory hooks.

#!/bin/bash

read oldrev newrev refname
short_refname=${refname##refs/heads/}

/usr/bin/git rev-list $oldrev...$newrev --no-merges --pretty=format:"\!dose ($short_refname): %s (%an) https://gforge.inria.fr/plugins/scmgit/cgi-bin/gitweb.cgi?p=dose/dose.git;a=commit;h=%h" --abbrev-commit | grep -v ^commit | while read a; do echo "$a" | cut -f-140 | bti --shrink-urls --logfile bti.log --config~/.bti_identica ; sleep 1; done

exit 0

your next commit should appear on identi.ca . The format string :

"\!dose ($short_refname): %s (%an) https://gforge.inria.fr/plugins/scmgit/cgi-bin/gitweb.cgi?p=dose/dose.git;a=commit;h=%h"

should print appear like this on identi.ca. Have a look at the man page of git rev-list if you want to add other info. Notice that the url gets shorten automatically and we use the !bang syntax to post in the group dose.

!dose (master): More work on the website (fix css issue) (Pietro Abate) http://identi.ca/url/73941892

For twitter is basically the same thing.

Managing the puppet manifest using a vcs is a best practice and there is a lot of material on the web. The easier way to do it, is to use git directly in the directory /etc/puppet and use a simple synchronization strategy with an external repo, either to publish your work, or simply to keep a backup somewhere.

Things are a bit more complicated when you would like to co-administer the machine with multiple people. Setting up user accounts, permission and everything can be a pain in the neck. Moreover working from your desktop is always more comfortable then logging in as root on a remote system and make changes there …

The solution I’ve chosen to make my life a bit easier is to use gitolite, that is a simple git gateway that works using ssh public keys for authentication and does not require the creating of local users on the server machine. Gitolite is available in debian and installing it is super easy : apt-get install gitolite .

If you use puppet already you might be tempted to use puppet to manage your gitolite installation. This is all good, but I don’t advice you to use modules like this one http://forge.puppetlabs.com/gwmngilfen/gitolite/1.0.0 as it’s going to install gitolite from source and on debian it’s not necessary … For my purposes, I didn’t find necessary to manage gitolite with puppet as all the default config options where good enough for me.

Once your debian package is installed, in order to initialize your repo, you just need to pass to gitolite the admin public key, that is your .ssh/id_rsa.pub key and then run this command:

sudo -H -u gitolite gl-setup /tmp/youruser.pub

This will create the admin and testing repo in /var/lib/gitolite/repositories and setup few other things. At this point you are ready to test you gitolite installation by cloning the admin repo :

git clone gitolite@example.org:gitolite-admin.git

Gitolite is engineered to use only the gitolite use to manage all your repositories. To add more repositories and users you should have a look at the documentation and then editing the file conf/gitolite.conf to add your new puppet repository.

At this point, you can go two ways. If you use git to manage your puppet directory, you can just make a copy of it somewhere and then add gitolite as a remote

git remote add origin gitolite@example.org:puppet.git

If you didn’t use git before, you can just copy the manifest in your new git repository, make a first commit and push it on the server.

git push origin master

Every authorized users can now use git to clone your puppet repository, hack, commit, push …

git clone gitolite@example.org:puppet

One last step is to add a small post-receive hook on the server to synchronize your gitolite repository with the puppet directory in /etc : This will sync your main puppet directory and trigger changes on the nodes for the next puppetd run. First I created a small shell script in /usr/local/bin/puppet-post-receive-hook.sh :

#!/bin/bash
umask 0022
cd /etc/puppet
git pull -q origin master

This script presuppose that your git repo in /etc/puppet has the gitolite repo as origin … Then I added a simple hook in the gitolite git repo that calls this script using sudo :

sudo /usr/local/bin/puppet-post-receive-hook.sh

And since you are at it you should also add a pre-commit hook to check the manifest syntax. This will save you a lot of useless commits.

If you use a more complicated puppet setup using environments (I’m not there yet, and I don’t think my setup will evolve in that direction in the near future), you can use puppet-sync that seems a neat script to do the job.

For the moment this setup works pretty well. I’m tempted to explore mcollective to trigger puppet runs on my nodes, but I’m there yet…

Update

There is an easier method to do all this using gbp-clone as described here. Ah !

Then to build the package, you just need to suggest git-buildpackage where to find the pristin-tar :

git-buildpackage --git-upstream-branch=upstream/master

or you could simply describe (as suggested) the layout in debian/gbp.conf.

Easy !!!

I’ve found a lot of different recipes and howtos about git debian packaging, but I failed to find one simple recipe to create a debian package from scratch when upstream is using git. Of course the following is a big patchwork from many different sources.

First we need to do a bit of administrative work to setup the repository :

mkdir yourpackage
cd yourpackage
git init --shared

Then, since I’m interested in tracking upstream development branch I’m going to add a remote branch to my repo:

git remote add upstream git://the.url/here.git

at this point I need to fetch upstream and create a branch for it.

git fetch upstream
git checkout -b upstream upstream/master

Now in my repo I have a master branch and an upstream branch. So far, so good. Let’s add the debian branch based on master:

git checkout master
git checkout -b debian master

It’s in the debian branch where I’m going to keep the debian related files. I’m finally read for hacking git add / git commit / git remove ...

When I’m done, I can switch to master, merge the debian branch into it and use git-buildpackage to build the package.

git checkout master
git branch 
  debian
* master
  upstream

git merge debian
git-buildpackage

Suppose I want to put everything on gitourious for example. I’ll create an acocunt, set up my ssh pub key and then I’ve to add an origin ref in my .git/config . Something like :

[remote "origin"]
       url = git@gitorious.org:debian-stuff/yourpackage.git
       fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
       remote = origin
       merge = refs/heads/master

The only thing left to do is to push everything on gitourious. the —all is important.

git push --all

People willing to pull your work from girourious have to follow the following script :

$git clone git@gitorious.org:debian-stuff/yourpackage.git
$cd yourpackage
$git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/debian
  remotes/origin/master
  remotes/origin/upstream
$git checkout -t origin/debian
$git checkout -t origin/upstream
$git branch -a
  debian
  master
* upstream
  remotes/origin/HEAD -> origin/master
  remotes/origin/debian
  remotes/origin/master
  remotes/origin/upstream
$git checkout master
$git-buildpackage

Maybe there is an easier way to pull all remote branches at once, but I’m not aware of it. Any better way ?

I’m definitely fed up with the lack of feature of svn. I always end up making backup copies of files, committing the wrong patch set, being unable to cherry-pick what to commit, not to talk about branching, merging and other marry activities. So today I overcome my laziness and setup a git svn repository for dose3 (since it’s all there, go ahead, and be happy!).

The concept of git svn if pretty easy. You work with git, and from time to time, you commit in svn. Best of both worlds, and ppl won’t even notice that you are not using their pet SCM, apart for the happy look of your face :)

First let’s clone the svn repo :

git svn clone https://gforge.info.ucl.ac.be/svn/mancoosi/trunk/dose3

This will probably hangup if you have big files in the svn repo. I don’t think it’s a problem with svn itself, but more likely a problem with the forge at ucl.ac.be. Anyway, if this happen, just go in the dose3 directory and finish it up with a git svn fetch

Now it’s time to sync all you changes and test from your old svn tree. I choose to rsync everything away :

rsync -avz --exclude=.svn --exclude=_build ../mancoosi-public/trunk/dose3/ .

of course we want to exclude all the compilation leftovers and .svn directories …

Ahhhhhhh freedom. Now I can edit, stash away changes, commit, uncommit, rebase, cherry-pick… I feel home.

At the end of the day we have to check if somebody updated the svn repo with git svn rebase, maybe resolve a couple of conflicts and then I can commit all my work with git svn dcommit.

I’ve been working with git svn only for a couple of days now, but I feel it pretty stable and trustworthy.

Note to Self

If you use github.com to host your projects these are two things to remember: * if you want a download url that is different from http://github.com/$user/$proj/tarball/master you can create a tag in your repo, push it to github and enjoy and new link in the download tab. Then you can download your project as http://github.com/$user/$proj/tarball/0.1 where 0.1 is the tag. Just for future ref : http://github.com/guides/push-tags-to-github

• If you want to create a debian package from a github project you should use the excellent service provided by Gunnar Wolf. For example, this is the redirect for my project ocaml-buddy : http://githubredir.debian.net/githubredir.cgi?author=abate&project=ocaml-buddy .Then your watch file will look like:

version=3
http://githubredir.debian.net/github/abate/ocaml-buddy (.*).tar.gz

Many other things to remember, I’m sure, but I forgot them all :)