.ocamlinit

Date Tags ocaml

When developing ocaml applications I often need to load a bunch of libraries in my toplevel to test a small function. This can be a tedious task when you have a complex project and a lot of libraries installed in your system. In the ocaml doco, kind of buried in the middle of a section there is the solution to this problem : .ocamlinit .

On start-up (before the first phrase is read), if the file .ocamlinit exists in the current directory, its contents are read as a sequence of Objective Caml phrases and executed as per the #use directive described in section 9.2. The evaluation outcode for each phrase are not displayed. If the current directory does not contain an .ocamlinit file, but the user’s home directory (environment variable HOME) does, the latter is read and executed as described below.

so for example if you often use ocamlfind-aware libraries and extlib in particular (I know, batteries… I know…) you can simple put these two lines in the .ocamlinit in your home directory.

#use "topfind";;
#require "extlib";;

and next time you fire the toplevel :

        Objective Caml version 3.11.2

Findlib has been successfully loaded. Additional directives:
  #require "package";;      to load a package
  #list;;                   to list the available packages
  #camlp4o;;                to load camlp4 (standard syntax)
  #camlp4r;;                to load camlp4 (revised syntax)
  #predicates "p,q,...";;   to set these predicates
  Topfind.reset();;         to force that packages will be reloaded
  #thread;;                 to enable threads

/usr/lib/ocaml/extlib: added to search path
/usr/lib/ocaml/extlib/extLib.cma: loaded
# 

As the documentation says you can have an .ocamlinit file in the directory of your project to load specific libraries.


bash option parsing

Date Tags bash

Since I know I’m going to forget how to do it properly, this is a tiny recipe to easily parse options in bash. I use getopts (and not getopt . !!! mind the S) that is a builtin shell command. help getopts to read the doco.

The snippet is pretty easy and self explanatory :

#!/bin/bash

usage()
{
DESCRIPTION=
cat << EOF
usage: $0 options

$DESCRIPTION

OPTIONS:
   -h      Show this message
   -v      Verbose
   -d      Debug
   -f      file
EOF
}

VERBOSE=
DEBUG=
FILE=
OTHERARGS=

while getopts "vhdf:" flag
do
  case "$flag" in
    f) FILE=$OPTARG ;;
    d) set -x ; DEBUG=true;;
    v) VERBOSE=true ;;
    h) usage ; exit 0 ;;
  esac
#  echo "$flag" $OPTIND $OPTARG
done

if [ -z $FILE ]; then
  shift $((OPTIND-1))
  OTHERARGS="$@"
fi

if [ -n "$VERBOSE"] ; then
  echo "verbose $VERBOSE"
  echo "debug $DEBUG"
  echo "file $FILE"
  echo "other args $OTHERARGS"
fi

exit 0

the Ocaml format module

Date Tags ocaml

Honestly ocaml format module is a royal PITA to use. The only documentation apart the reference manual is this document here. Don’t get me wrong. I think it’s a very nice piece of software and absolutely worth having it in the stdlib, but it simply not intuitive (at least for me) to use at the first glance. I’ll write down a couple of example. hopefully this will help me - and others - the next time I’ll need to use it.

I’m going to use the Format.fprintf function quite a lot. This function uses similar formatting string to the more widely used Printf.fprintf. In the Format module page you can find all the details. Let’s start easy and print a string. We write a pretty printer function pp_cell that gets a formatter and an element. This is my favourite way of writing printing function as I can daisy chain together in a printf function call using the "%a" formatting string. If the formatter is Format.std_formatter the string will be printed on stdout.

let pp_cell fmt cell = Format.fprintf fmt "%s" cell

Next we examine a simple function to pretty printer a list of elements. The signature of this function is quite similar as before, but this time we also pass an optional separator and a pretty printer for the element of the string.

let rec pp_list ?(sep="") pp_element fmt = function
  |[h] -> Format.fprintf fmt "%a" pp_element h
  |h::t ->
      Format.fprintf fmt "%a%s@,%a"
      pp_element h sep (pp_list ~sep pp_element) t
  |[] -> ()

The function takes care of printing the separator after all elements but the last one.

Let’s start playing witht the boxes. The formatting boxes are the main reason why I use the format module and they are very handy if you want to pretty print nested structure easily.

If we use the std_formatter and the list pretty printer without formatting box, we obtain this output.

# let fmt = Format.std_formatter ;;
# (pp_list ~sep:"," pp_cell) fmt ["aa";"bb";"cc"];;
aa,bb,
cc- : unit =
# 

that is the same as :

# Format.fprintf fmt "%a" (pp_list ~sep:"," pp_cell) ["aa";"bb";"cc"];;
aa,bb,
cc- : unit = ()

To be frank, I don’t quite get yet why the formatter decide to add a new line after the last comma… but moving on. If I now use a formatting box, the result is different. To print the list one one line, I can use the hbox. If I want a vertical list, I can use the vbox. This gives respectively:

# Format.fprintf fmt "@[<h>%a@]@." (pp_list ~sep:"," pp_cell) ["aa";"bb";"cc"];;
aa,bb,cc
# Format.fprintf fmt "@[<v>%a@]@." (pp_list ~sep:"," pp_cell) ["aa";"bb";"cc"];;
aa,
bb,
cc

If we want to print a list with one character of indentation, this can be easily done as:

Format.fprintf fmt "@[<v 1>@,%a@]@." (pp_list ~sep:"," pp_cell) ["aa";"bb";"cc"];;
 aa,
 bb,
 cc

The idea is that by changing the type of formatting boxes, the soft break @, is interpreted differently by the formatter, once as newline, once as space. Moreover by adding an indentation, the formatter will take care of adding an offset to all strings printed within that box. And this is a winner when pretty printing nested structures.

Lets now delve a bit deeper and let’s try to format a table… I didn’t found any tutorial on the net about this, but bit and pieces of code buried into different projects… A table for me is a tuple composed by a header (a string array) and two-dimensional array string array. The point here is to format the table in a way where each element is displayed in a column in relation to the longest element in the table. First we need two support pretty printers, one for the header and the other one the each row in the table. In order to set the tabulation margins of the table, we need to find, for each column the longest string in the table. The result of this computation (the function is shown below in pp_table) is an array of integer widths. When we print the header of the table, we make sure to set the width of each column with the Format.pp_set_tab fmt function. The magic of the Format module will take care of the rest. The second function to print each row is pretty straightforward to understand.

let pp_header widths fmt header =
  let first_row = Array.map (fun x -> String.make (x + 1) ' ') widths in
  Array.iteri (fun j cell ->
    Format.pp_set_tab fmt ();
    for z=0 to (String.length header.(j)) - 1 do cell.[z] <- header.(j).[z] done;
    Format.fprintf fmt "%s" cell
  ) first_row

let pp_row pp_cell fmt row =
  Array.iteri (fun j cell ->
    Format.pp_print_tab fmt ();
    Format.fprintf fmt "%a" pp_cell cell
  ) row

The pretty printer for the table is pretty easy now. First we compute the width of the table, then we open the table box, we print the headers, we iterate on each row and we close the box. tadaaaa :)

let pp_tables pp_row fmt (header,table) =
  (* we build with the largest length of each column of the 
   * table and header *)
  let widths = Array.create (Array.length table.(0)) 0 in
  Array.iter (fun row ->
    Array.iteri (fun j cell ->
      widths.(j) <- max (String.length cell) widths.(j)
    ) row
  ) table;
  Array.iteri (fun j cell ->
    widths.(j) <- max (String.length cell) widths.(j)
  ) header;

  (* open the table box *)
  Format.pp_open_tbox fmt ();

  (* print the header *)
  Format.fprintf fmt "%a@\n" (pp_header widths) header;
  (* print the table *)
  Array.iter (pp_row fmt) table;

  (* close the box *)
  Format.pp_close_tbox fmt ();

for example this is what we get :

let a = Array.make_matrix 3 4 "aaaaaaaa" in
let h = Array.make 4 "dddiiiiiiiiiiiiiiiii" in
let fmt = Format.std_formatter in
Format.fprintf fmt "%a" (pp_tables (pp_row pp_cell)) (h,a);;
dddiiiiiiiiiiiiiiiii          dddiiiiiiiiiiiiiiiii          dddiiiiiiiiiiiiiiiii           dddiiiiiiiiiiiiiiiii 
aaaaaaaa             aaaaaaaa             aaaaaaaa             aaaaaaaa
aaaaaaaa             aaaaaaaa             aaaaaaaa             aaaaaaaa
aaaaaaaa             aaaaaaaa             aaaaaaaa             aaaaaaaa

Well … more or less. On the terminal you will notice that everything is well aligned. This is of course only to scratch the surface. There are still few things I don’t really understand, and many functions that I didn’t consider at all. Maybe I’ll write a second chapter one day.


redmine 1.0.0

A long while ago I wrote about installing redmine as a normal user on debian. I’ve been using and administering a redmine installation for quite some time now and I’m very happy with it. Since the new stable release of redmine was released in june, I decided to give it another try.

The redmine installation manual is already very detailed. These are the steps I followed to get a simple redmine instance running on my system. First we need to install a specific version of rails and rack. I didn’t check which version we have in debian (sid in this example), but since the gem system is very easy to use, I prefer to stay on the safe side and to use the recommended versions. To install the ruby gems is an arbitrary directory it is sufficient to set the GEM_PATH environment variable. Everything else follows pretty easily.

export GEM_PATH=`pwd`/gems
gem install -i $GEM_PATH rails -v=2.3.5
gem install -i $GEM_PATH rack -v=1.0.1

Since I don’t want to bother with a full fledged database, in this example I’ll use the sqlite3 backend. In order to do this, I need to install another gem :

gem install -i $GEM_PATH sqlite-ruby

I’m ready to get the redmine code. I prefer to get it from the svn directly, so I conveniently can keep it up-to-date.

svn co http://redmine.rubyforge.org/svn/branches/1.0-stable redmine-1.0

Now we need to configure the database backend, This is accomplished just be adding the following two line to the file config/database.yml. Mind that the backend is called sqlite and not sqlite3 !

production:
  adapter: sqlite
  database: db/test.db

Time to create and initialize the db with the default data. These two commands will create the db structure and add trackers, default users and take care of other details.

RAILS_ENV=production rake db:migrate
RAILS_ENV=production rake redmine:load_default_data

Since we are going to run the redmine instance either as a fcgi or with the standalone server, we just need to make sure that few directories are writable to my user :

chmod -R 755 files log tmp public/plugin_assets

Time to try it out !

ruby script/server webrick -e production

and to point our favorite browser to http://localhost:3000/

DONE!

Last time I tried redmine I had to fight a bit to install the ldap authentication plugin. Just by looking around the new stable version, I’m pretty happy to notice that it is now part of the default bundle. This is nice ! I’ve left to setup the fastcgi server and to configure the SCM repositories. Maybe I’ll write a part two later…


edos-distcheck - new YAML output format

During the last two days I spent some time to implement part of the [http://wiki.debian.org/EDOS/ProposalDose3 proposed features for distcheck/ edos-distcheck]. Since everybody is at debconf and talk is silver, but code is gold, I hope that a real implementation can get the ball rolling and get us closer to a stable release of the next generation of edos/mancoosi tools.

In particular this post is about the new YAML output format for distcheck. The rational to use YAML is to have a data structure that is at the same time human and machine friendly. There are a lot of scripts in debian that rely on distcheck and we want to provide a grep friendly output that sat the same time doesn’t hurt your eyes. The other proposed solution was to use json, but was ditched in favor of YAML. We also removed the xml output.

In order to provide a machine readable output and to minimize parsing mistakes, I used the schema language proposed here. This is the resulting data structure definition :

type: seq
sequence:
  - type: map
    mapping:
      "package": { type: str, required: true }
      "version": { type: text, required: true }
      "status":  { type: str, enum: [ broken, ok ], required: true }
      "installationset":
         type: seq
         sequence:
           - type: map
             mapping:
               "package": { type: str, required: true }
               "version": { type: text, required: true }
      "reasons":
         type: seq
         sequence:
           - type: map
             mapping:
               "conflict":
                  type: map
                  mapping:
                    "pkg1":
                      type: map
                      required: true
                      mapping:
                        "package": { type: str, required: true }
                        "version": { type: text, required: true }
                    "pkg2":
                      type: map
                      required: true
                      mapping:
                        "package": { type: str, required: true }
                        "version": { type: text, required: true }
                    "paths":
                      type: seq
                      sequence:
                        - type: map
                          mapping:
                            "path":
                              type: seq
                              sequence:
                                - type: map
                                  mapping:
                                    "package": { type: str, required: true }
                                    "version": { type: text, required: true }
                                    "vpkg": { type: str, required: false}
               "missing":
                  type: map
                  mapping:
                    "pkg":
                      type: map
                      required: true
                      mapping:
                        "package": { type: str, required: true }
                        "version": { type: text, required: true }
                        "vpkg": { type: str, required: false}
                    "paths":
                      type: seq
                      sequence:
                        - type: map
                          mapping:
                            "path":
                              type: seq
                              sequence:
                                - type: map
                                  mapping:
                                    "package": { type: str, required: true }
                                    "version": { type: text, required: true }
                                    "vpkg": { type: str, required: false}

There are few improvements on the old output from edos-distcheck. We are going to discuss these with a real example. The following tow snippets are from the output of distcheck on sid/amd64 (04/08/2010).

Distcheck now outputs a list of broken or installable packages depending on the given options (—failoures , —success, —explain and combinations of thereof ) . Two quick examples :

-
 package: python-gi-dbg
 version: 0.6.0-1
 status: broken
 reasons:
  -
   conflict:
    pkg1:
     package: python-gobject
     version: 2.21.4-1
    pkg2:
     package: python-gi
     version: 0.6.0-1
    paths:
     -
      path:
       -
        package: python-gi-dbg
        version: 0.6.0-1
        vpkg: python-gi (= 0.6.0-1)
       -
        package: python-gi
        version: 0.6.0-1
     -
      path:
       -
        package: python-gi-dbg
        version: 0.6.0-1
        vpkg: python-gi (= 0.6.0-1)
       -
        package: python-gi
        version: 0.6.0-1
        vpkg: python-gobject (>= 2.20)
       -
        package: python-gobject
        version: 2.21.4-1

In the example above, the package python-gi-dbg is broken because there is a conflict between the packages python-gobject and python-gi. The reason why python-gi-dbg is affected by this conflict is explained by following the dependency chain from python-gi-dbg to the two offending packages. Note that for each package element of each path we specify the vpkg, that is the dependency (as it is written in the control file) that lead to the conflict. Since a dependency can be a virtual package or a package with a version constraint, it can be expanded to a disjunction of packages (think a dependency on mta-agent can be expanded as postfix. exim or sendmail…). All possible paths to an offending package are reported.

Likewise if a package is broken because there is an unfulfilled dependency, distcheck will show the path leading to the problem . In the following example we show that the package gnash-tools is broken because there are two dependency that depend on the missing package libboost-date-time1.40.0 (>= 1.40.0-1).

-
 package: gnash-tools
 version: 0.8.7-2+b1
 status: broken
 reasons:
  -
   missing:
    pkg:
     package: gnash-common
     version: 0.8.7-2+b1
     vpkg: libboost-date-time1.40.0 (>= 1.40.0-1)
    paths:
     -
      path:
       -
        package: gnash-tools
        version: 0.8.7-2+b1
        vpkg: gnash-common-opengl (= 0.8.7-2+b1) | gnash-common (= 0.8.7-2+b1)
       -
        package: gnash-common
        version: 0.8.7-2+b1
        vpkg: libboost-date-time1.40.0 (>= 1.40.0-1)
  -
   missing:
    pkg:
     package: gnash-common-opengl
     version: 0.8.7-2+b1
     vpkg: libboost-date-time1.40.0 (>= 1.40.0-1)
    paths:
     -
      path:
       -
        package: gnash-tools
        version: 0.8.7-2+b1
        vpkg: gnash-common-opengl (= 0.8.7-2+b1) | gnash-common (= 0.8.7-2+b1)
       -
        package: gnash-common-opengl
        version: 0.8.7-2+b1
        vpkg: libboost-date-time1.40.0 (>= 1.40.0-1)

The code is still in a flux and it is not ready for production yet (everything is in the mancoosi svn). I hope this is a good step in the right direction. Comments on the debian wiki are welcome.

if we compare the output of distcheck with the old edos-debcheck we get the following:

$cat /var/lib/apt/lists/ftp.debian.org_debian_dists_unstable_main_binary-amd64_Packages | edos-debcheck -failures -explain
[...]
holotz-castle-milanb (= 0.0.20050210-1): FAILED
  holotz-castle-milanb (= 0.0.20050210-1) depends on one of:
  - holotz-castle (= 1.3.14-2)
  holotz-castle-data (= 1.3.14-2) and holotz-castle-milanb (= 0.0.20050210-1) conflict
  holotz-castle (= 1.3.14-2) depends on one of:
  - holotz-castle-data (= 1.3.14-2)
$./debcheck --explain --failures /var/lib/apt/lists/ftp.debian.org_debian_dists_unstable_main_binary-amd64_Packages
[...]
-
 package: holotz-castle-milanb
 version: 0.0.20050210-1
 status: broken
 reasons:
  -
   conflict:
    pkg1:
     package: holotz-castle-milanb
     version: 0.0.20050210-1
    pkg2:
     package: holotz-castle-data
     version: 1.3.14-2
    paths:
     -
      path:
       -
        package: holotz-castle-milanb
        version: 0.0.20050210-1
        vpkg: holotz-castle
       -
        package: holotz-castle
        version: 1.3.14-2
        vpkg: holotz-castle-data (= 1.3.14-2)
       -
        package: holotz-castle-data
        version: 1.3.14-2