Here at mancoosi we have been working for quite a while to promote and advance solver technology for FOSS distributions. We are almost at the end of the project and it is important to make the mancoosi technology relevant for the community. On goal of the project is to provide a prototype that uses part of the results of mancoosi that can based to install/remove/upgrade packages on a user machine. We certainly don’t want to create yet another meta installer. This would be very time consuming and certainly going beyond the scope of the project. The idea is to create a prototype, that can work as an apt-get drop in replacement that will allow everybody to play with different solvers and installation criteria.
A very first integration step is a small shell script apt-mancoosi that tries to put together different tools that we have implemented during the project. Roberto wrote extensively about his experience with apt-mancoosi a while ago showing that somehow the mancoosi tools are already usable, as proof of concept, to experiment with all solvers participating to the Misc competition.
On notable obstacle we encountered with apt-mancoosi is how to pipe the result of an external solver to apt-get to effectively install the packages proposed as solution. Apt-mancoosi fails to be a drop-in replacement for apt-get exactly for this reason.
The “problem” is quite simple : The idea at the beginning was to pass to apt-get, on the command line, a request that effectively represents a complete solution. We expected that, since this was already a locked-down solution, apt-get would have just installed all packages without any further modification to the proposed installation set. Of course, since apt-get is designed to satisfy a user request, and not just to install packages, we quickly realized that our evil plan was doomed to failure.
The only option left, was to use libapt directly, but the idea of programming in c++ quickly made me to desist. After a bit of research (not that much after all), I finally found a viable solution to our problems in python-apt that is a low level and wrapper around libapt. This definitely made my day.
Now the juicy details. the problem was to convince apt to completely bypass the solver part and just call the installer. First a small intro. python-apt has an extensive documentation with a couple of tutorials. Using python-apt is actually pretty easy (some snippet from the python-apt doco) :
import apt
# First of all, open the cache
cache = apt.Cache()
# Now, lets update the package list
cache.update()
here we open the cache (apt.Cache is a wrapper around the low level binding in the apt_pkg module), then we update the package list. This is equivalent to apt-get update . Installing a package is equally easy :
import apt
cache = apt.Cache()
pkg = cache['python-apt']
# Mark python-apt for install
pkg.mark_install()
# Now, really install it
cache.commit()
Now, the method mark_install of the module package will effectively run the solver to resolve and mark all the dependencies of the package python-apt. This is the default behavior when apt-get is used on the command line. This method has however three optional arguments that are just what I was looking for, that is autoFix, autoInst and fromUser . The explanation from the python-apt doco is quite clear.
mark_install(*args, **kwds)
Mark a package for install.
If autoFix is True, the resolver will be run, trying to fix broken packages. This is the default.
If autoInst is True, the dependencies of the packages will be installed automatically. This is the default.
If fromUser is True, this package will not be marked as automatically installed. This is the default. Set it to False if you want to be able to automatically remove the package at a later stage when no other package depends on it.
What we want is to set autoFix and autoInst to false to completely
bypass the solver. So imagine that an external solver can give use a
string of the form :
bash+ dash=1.4 baobab-
that basically asks to install bash to the newest version, dash at
version 1.4 and remove baobab. Suppose also that this is a complete
solution, that is, all dependencies are satisfied and there are no conflicts.
The work flow of mpm (mancoosi package manager) is as follows : init apt-get, convert all packages lists + status in a cudf description, pass this cudf to an external solver, get the result and set all packages to add/remove in the apt.cache of python-apt download the packages commit the changes (effectively, run dpkg)
We already have a first prototype on the mancoosi svn. It’s not released yet as we are waiting to do more testing, add more options and make it stable enough for testing. Maybe one day, this will be uploaded to debian.
This is the trace of a successful installation of a package in a lenny chroot. The solver used here is the p2 solver
dev:~/mpm# ./mpm.py -c apt.conf install baobab
Running p2cudf-paranoid-1.6 solver ...
Validate solution ...
loading CUDF ...
loading solution ...
Summary of proposed changes:
new: 30
removed: 0
replaced: 0
upgraded: 0
downgraded: 0
unsatisfied recommends:: 8
changed: 30
uptodate: 322
notuptodate: 116
New packages: baobab (2.30.0-2) dbus-x11 (1.2.24-3) gconf2 (2.28.1-5)
gconf2-common (2.28.1-5) gnome-utils-common (2.30.0-2)
libatk1.0-0 (1.30.0-1) libcairo2 (1.8.10-6) libdatrie1 (0.2.4-1)
libdbus-glib-1-2 (0.88-2) libgconf2-4 (2.28.1-5) libgtk2.0-0 (2.20.1-2)
libgtk2.0-common (2.20.1-2) libgtop2-7 (2.28.1-1) libgtop2-common (2.28.1-1)
libidl0 (0.8.14-0.1) libjasper1 (1.900.1-7+b1) liborbit2 (1:2.14.18-0.1)
libpango1.0-0 (1.28.3-1) libpango1.0-common (1.28.3-1)
libpixman-1-0 (0.16.4-1) libthai-data (0.1.14-2) libthai0 (0.1.14-2)
libtiff4 (3.9.4-5) libxcb-render-util0 (0.3.6-1) libxcb-render0 (1.6-1)
libxcomposite1 (1:0.4.2-1) libxcursor1 (1:1.1.10-2) libxrandr2 (2:1.3.0-3)
psmisc (22.11-1) shared-mime-info (0.71-3)
Removed packages:
Replaced packages:
Upgraded packages:
Selecting previously deselected package libatk1.0-0.
(Reading database ... 28065 files and directories currently installed.)
Unpacking libatk1.0-0 (from .../libatk1.0-0_1.30.0-1_i386.deb) ...
Selecting previously deselected package libpixman-1-0.
Unpacking libpixman-1-0 (from .../libpixman-1-0_0.16.4-1_i386.deb) ...
Selecting previously deselected package libxcb-render0.
Unpacking libxcb-render0 (from .../libxcb-render0_1.6-1_i386.deb) ...
Selecting previously deselected package libxcb-render-util0.
Unpacking libxcb-render-util0 (from .../libxcb-render-util0_0.3.6-1_i386.deb) ...
Selecting previously deselected package libcairo2.
Unpacking libcairo2 (from .../libcairo2_1.8.10-6_i386.deb) ...
Selecting previously deselected package libdbus-glib-1-2.
Unpacking libdbus-glib-1-2 (from .../libdbus-glib-1-2_0.88-2_i386.deb) ...
Selecting previously deselected package libidl0.
Unpacking libidl0 (from .../libidl0_0.8.14-0.1_i386.deb) ...
Selecting previously deselected package liborbit2.
Unpacking liborbit2 (from .../liborbit2_1%3a2.14.18-0.1_i386.deb) ...
Selecting previously deselected package gconf2-common.
Unpacking gconf2-common (from .../gconf2-common_2.28.1-5_all.deb) ...
Selecting previously deselected package libgconf2-4.
Unpacking libgconf2-4 (from .../libgconf2-4_2.28.1-5_i386.deb) ...
Selecting previously deselected package libgtk2.0-common.
Unpacking libgtk2.0-common (from .../libgtk2.0-common_2.20.1-2_all.deb) ...
Selecting previously deselected package libjasper1.
Unpacking libjasper1 (from .../libjasper1_1.900.1-7+b1_i386.deb) ...
Selecting previously deselected package libpango1.0-common.
Unpacking libpango1.0-common (from .../libpango1.0-common_1.28.3-1_all.deb) ...
Selecting previously deselected package libdatrie1.
Unpacking libdatrie1 (from .../libdatrie1_0.2.4-1_i386.deb) ...
Selecting previously deselected package libthai-data.
Unpacking libthai-data (from .../libthai-data_0.1.14-2_all.deb) ...
Selecting previously deselected package libthai0.
Unpacking libthai0 (from .../libthai0_0.1.14-2_i386.deb) ...
Selecting previously deselected package libpango1.0-0.
Unpacking libpango1.0-0 (from .../libpango1.0-0_1.28.3-1_i386.deb) ...
Selecting previously deselected package libtiff4.
Unpacking libtiff4 (from .../libtiff4_3.9.4-5_i386.deb) ...
Selecting previously deselected package libxcomposite1.
Unpacking libxcomposite1 (from .../libxcomposite1_1%3a0.4.2-1_i386.deb) ...
Selecting previously deselected package libxcursor1.
Selecting previously deselected package libxrandr2.
Unpacking libxrandr2 (from .../libxrandr2_2%3a1.3.0-3_i386.deb) ...
Selecting previously deselected package shared-mime-info.
Unpacking shared-mime-info (from .../shared-mime-info_0.71-3_i386.deb) ...
Selecting previously deselected package libgtk2.0-0.
Unpacking libgtk2.0-0 (from .../libgtk2.0-0_2.20.1-2_i386.deb) ...
Selecting previously deselected package libgtop2-common.
Unpacking libgtop2-common (from .../libgtop2-common_2.28.1-1_all.deb) ...
Selecting previously deselected package libgtop2-7.
Unpacking libgtop2-7 (from .../libgtop2-7_2.28.1-1_i386.deb) ...
Selecting previously deselected package psmisc.
Unpacking psmisc (from .../psmisc_22.11-1_i386.deb) ...
Selecting previously deselected package dbus-x11.
Unpacking dbus-x11 (from .../dbus-x11_1.2.24-3_i386.deb) ...
Selecting previously deselected package gconf2.
Unpacking gconf2 (from .../gconf2_2.28.1-5_i386.deb) ...
Selecting previously deselected package gnome-utils-common.
Unpacking gnome-utils-common (from .../gnome-utils-common_2.30.0-2_all.deb) ...
Selecting previously deselected package baobab.
Unpacking baobab (from .../baobab_2.30.0-2_i386.deb) ...
Processing triggers for man-db ...
Setting up libatk1.0-0 (1.30.0-1) ...
Setting up libpixman-1-0 (0.16.4-1) ...
Setting up libxcb-render0 (1.6-1) ...
Setting up libxcb-render-util0 (0.3.6-1) ...
Setting up libcairo2 (1.8.10-6) ...
Setting up libdbus-glib-1-2 (0.88-2) ...
Setting up libidl0 (0.8.14-0.1) ...
Setting up liborbit2 (1:2.14.18-0.1) ...
Setting up gconf2-common (2.28.1-5) ...
Creating config file /etc/gconf/2/path with new version
Setting up libgconf2-4 (2.28.1-5) ...
Setting up libgtk2.0-common (2.20.1-2) ...
Setting up libjasper1 (1.900.1-7+b1) ...
Setting up libpango1.0-common (1.28.3-1) ...
Cleaning up font configuration of pango...
Updating font configuration of pango...
Cleaning up category xfont..
Updating category xfont..
Setting up libdatrie1 (0.2.4-1) ...
Setting up libthai-data (0.1.14-2) ...
Setting up libthai0 (0.1.14-2) ...
Setting up libpango1.0-0 (1.28.3-1) ...
Setting up libtiff4 (3.9.4-5) ...
Setting up libxcomposite1 (1:0.4.2-1) ...
Setting up libxcursor1 (1:1.1.10-2) ...
Setting up libxrandr2 (2:1.3.0-3) ...
Setting up shared-mime-info (0.71-3) ...
Setting up libgtk2.0-0 (2.20.1-2) ...
Setting up libgtop2-common (2.28.1-1) ...
Setting up libgtop2-7 (2.28.1-1) ...
Setting up psmisc (22.11-1) ...
Setting up dbus-x11 (1.2.24-3) ...
Setting up gconf2 (2.28.1-5) ...
update-alternatives: using /usr/bin/gconftool-2 to provide /usr/bin/gconftool (gconftool) in auto mode.
Setting up gnome-utils-common (2.30.0-2) ...
Setting up baobab (2.30.0-2) ...
Broken: 0
InstCount: 30
DelCount: 0
dev:~/mpm#
I think we’ll keep working on this python prototype for a while, but this is not certainly what we want to propose to the community. The mancoosi package manager is probably going to be written in Ocaml and integrated with dose3 and libcudf. This will allow us to gain speed and have a solid language to develop with (nothing against python, but we don’t feel that a scripting language is suitable for an essential component as a package manager). Time will tell. For the moment this is just vapor-ware …