4. The pbuilder Backport Process

Successfully backporting a typical package is often a successive series of individual backporting activities, accumulating a set of packages that fulfill a dependency tree.

The usual process begins with identifying the distribution containing the version of the package you require. Once you have found the version you require, the package source is retrieved. Next pbuilder is run and it walks through the dependency tree for you, informing you of any missing dependencies for building. If there are none, the package is built and the result written to /var/cache/pbuilder/result. Otherwise, the missing dependency is pushed onto the top of the stack of packages needed to successfully backport your goal package.

4.1. Process Overview

In its simplest form, the process flows like the example below. pbuilder produces a substantial amount of output, much of which will be discussed in detail later.


$ mkdir -p ~/src/debian/wget
$ cd ~/src/debian/wget
$ apt-get source wget
Reading Package Lists... Done
Building Dependency Tree... Done
Need to get 1226kB of source archives.
Get:1 http://http.us.debian.org/debian unstable/main wget 1.10.2-1 (dsc) [609B]
Get:2 http://http.us.debian.org/debian unstable/main wget 1.10.2-1 (tar) [1213kB]
Get:3 http://http.us.debian.org/debian unstable/main wget 1.10.2-1 (diff) [12.2kB]
Fetched 1226kB in 50s (24.3kB/s)
dpkg-source: extracting wget in wget-1.10.2
$ ls -1
wget-1.10.2
wget_1.10.2-1.diff.gz
wget_1.10.2-1.dsc
wget_1.10.2.orig.tar.gz

As demonstrated above, it is preferable to create a new directory for each backport project which will contain all the files for the task at hand.


$ sudo pbuilder build wget_1.10.2-1.dsc
W: /home/jasonb/.pbuilderrc does not exist
I: using fakeroot in build.
pbuilder-buildpackage/i386 $Id: pbuilder-buildpackage-funcs,
 v 1.28 2005/12/21 11:57:29 dancer Exp $
$ Id: pbuilder-buildpackage,v 1.122 2006/03/17 22:17:16 dancer Exp $

Current time: Wed Apr 19 04:25:46 UTC 2006
pbuilder-time-stamp: 1145420746
Building the build Environment
 -> extracting base tarball [/var/cache/pbuilder/base.tgz]
...
 -> user script /var/cache/pbuilder/build//28826/tmp/hooks/D01apt-ftparchive finished
 -> Attempting to parse the build-deps : pbuilder-satisfydepends,
 v 1.23 2006/02/11 13:17:04 dancer Exp $
 -> Considering  debhelper (>> 3.0.0)
...
Copying back the cached apt archive contents
/var/cache/pbuilder/build/28826/etc/passwd
/var/cache/pbuilder/build/28826/etc/group
Copying source file
    -> copying [wget_1.10.2-1.dsc]
    -> copying [./wget_1.10.2.orig.tar.gz]
    -> copying [./wget_1.10.2-1.diff.gz]
Extracting source
...
dpkg-source: extracting wget in wget-1.10.2
 -> Building the package
W: no hooks of type A found -- ignoring

Next, the build process begins. pbuilder will spring to life, generating a proper build environment. Once that is complete, it will attempt to satisfy the build dependencies for the package in question, wget in this instance. You will find this is the stage that will most often fail, requiring you to backport dependency packages, too.

If all dependencies can be satisfied, pbuilder will attempt to build the package using dpkg-buildpackage. Now, the actual build process may fail at any point as a result of a configure error, a compile error, or a bug in the package building helper scripts.


dpkg-deb: building package `wget' in `../wget_1.10.2-1_i386.deb'.
W: no hooks of type B found -- ignoring
Copying back the cached apt archive contents
 -> unmounting dev/pts filesystem
 -> unmounting proc filesystem
 -> unmounting /var/cache/pbuilder/result filesystem
Current time: Wed Apr 19 04:07:10 UTC 2006
pbuilder-time-stamp: 1145419630
 -> cleaning the build env
    -> removing directory /var/cache/pbuilder/build//19244 and its subdirectories

Fortunately, as shown above, the build process for the wget package was successful and no additional dependencies were necessary. If only all packages built so easily, no?

4.2. Preparing to Backport

Oops. slib would be a better example, as it depends on scm which isn't in Stable. Then, you have to backport scm, too. It'll look mostly like this, but need the slib build-dep failure, too. Oh, nice it conflicts with Stable's libguile, so this is a great example. Redo.

Now that a functional pbuilder environment is configured, it's time to attempt a backport. The first step is identifying what you want to backport. For example, let's say we want to backport the package curl to continue our theme of command line based HTTP aware clients.

In order to obtain the source files for curl, the easiest approach is to modify our /etc/apt/sources.list to include a source line for Debian Testing so we can obtain a recent copy of scm.


# perl -i -pe 's/^(deb-src.*)/#$1/' /etc/apt/sources.list
# cat<<EOF > /etc/apt/sources.list
deb-src http://http.us.debian.org/debian/ testing main contrib non-free
EOF
# apt-get update

As demonstrated above, it's important to ensure only the deb-src we need is the only one uncommented so we download the correct source files. You may find packages.debian.org helpful in determining which Debian distribution to pull the source package from. Next, let's download.


$ mkdir -p ~/src/debian/curl
$ cd ~/src/debian/scm
$ apt-get source curl
Reading Package Lists... Done
Building Dependency Tree... Done
Need to get 1837kB of source archives.
Get:1 http://http.us.debian.org testing/main curl 7.15.3-1 (dsc) [940B]
Get:2 http://http.us.debian.org testing/main curl 7.15.3-1 (tar) [1819kB]
Get:3 http://http.us.debian.org testing/main curl 7.15.3-1 (diff) [17.5kB]
Fetched 1837kB in 12s (152kB/s)
dpkg-source: extracting curl in curl-7.15.3
$ ls -1
curl-7.15.3
curl_7.15.3-1.diff.gz
curl_7.15.3-1.dsc
curl_7.15.3.orig.tar.gz

Now we're ready to begin backporting our first package.

4.3. Dependency Resolution

While we could look at each dependency by hand, we would be missing out on one of pbuilder's useful features, automated dependency resolution.

Below, we will see pbuilder resolve each dependency. If a dependency cannot be satisfied, it's generally because a newer package than is available is required or a required package does not exist in your release of Debian Stable. Additionally, during the actual package build phase, an unlisted dependency may cause the build to fail. Such a failure should be filed in Debian's BTS.


$ sudo pbuilder build curl_7.15.3-1.dsc 
W: /.pbuilderrc does not exist
I: using fakeroot in build.
pbuilder-buildpackage/i386 $Id: pbuilder-buildpackage-funcs,
 v 1.28 2005/12/21 11:57:29 dancer Exp $
$ Id: pbuilder-buildpackage,v 1.122 2006/03/17 22:17:16 dancer Exp $

Current time: Mon May 15 23:11:41 EDT 2006
...
 -> Attempting to parse the build-deps :
 pbuilder-satisfydepends,v 1.24 2006/03/31 00:14:31 dancer Exp $
 -> Considering  debhelper (>> 5)
   -> Trying debhelper
 -> Considering  autoconf
   -> Trying autoconf
 -> Considering  libtool
   -> Trying libtool
 -> Considering  automake1.7
   -> Trying automake1.7
 -> Considering  binutils (>= 2.14.90.0.7)
   -> Trying binutils
 -> Considering  libssl-dev (>= 0.9.8-1)
      Tried versions: 0.9.7e-3sarge1
   -> Does not satisfy version, not trying
E: Could not satisfy build-dependency.
E: pbuilder-satisfydepends failed.

A disappointing but frequent development, we have discovered our progress has been stopped by a failed build dependency for curl It seems the maintain requires a newer version of libssl-dev than we have available to us. It's noted above that of the versions available, only one in this case, its version is too low. (Technically debhelper (>> 5) is not in Debian Stable either, but we're going to side step that by using Debian Backports.)

In order to resolve this we have two realistic options available to us. First, we can simply try encouraging the package to build against the available dependencies. It may be the case that curl simply needs some version of libssl-dev and not necessarily the specific version described in debian/control, the source package's control file. Second, we can backport the required dependency, then try building curl again.

4.3.1. Overriding Package Dependencies

Depending on your familiarity with the package in question, you may feel comfortable trumping the official package maintainer. Let's see what happens.

First, let us move the existing, unpacked source tree and modify it.


$ mv curl-7.15.3/ curl-7.15.3.jab
$ cd curl-7.15.3.jab/debian/
$ vi control

Now, let us change the dependency for libssl-dev to a version we can satisfy without any additional backporting effort on our part. Find the line with Build-Depends, generally all on a single line. While maintaining the line formatting, let us change the the dependency from the former to the latter below.


Build-Depends: debhelper (>> 5), ..., libssl-dev (>= 0.9.8-1), ...
Build-Depends: debhelper (>> 5), ..., libssl-dev (>= 0.9.7e-3sarge1), ...

Additionally, you will find some debian/control Package sections that contain install dependencies in the form of Depends. While this package does not demonstrate it, for some backports you may need to tweak the Depends field in the same manner as the Build-Depends field, but using a package's library package name and not its development package name.

Finally, we need to bump the package revision such that our new build is unique. We will use dch for this, although it's entirely possible to edit debian/changelog by hand. Though initially strange, the official package revisions are maintained in debian/changelog. By default, dch will invoke your favorite editor using the EDITOR environment variable.


$ dch -i
curl (7.15.3-2) unstable; urgency=low

  *

 -- Jason Boxman <jasonb@edseek.com>  Mon, 15 May 2006 23:32:33 -0400

Above, you will see we're given a fairly clean slate. dch will guess a maintainer name, which you can override, a version bumped by one, and the current date and time. The version chosen is fairly important, so let us examine that first.

If we go with the default of 7.15.3-2, a future version released upstream with that same revision won't be installable. Instead, it's often best to choose a slightly lower version number. Based on how version strings are evaluated in Debian, 7.15.3-1jab1 is more appropriate. It will evaluate as newer than the original version 7.15.3-1, but older than any possible future 7.15.3-2. The underlying assumption is that, eventually, we want to return to the fold and upgrade to the next release of Debian Stable. A different versioning scheme for backports would be appropriate if the intention is to fork one or more packages permanently.

With that out of the way, let us add a simple description of what we did and save the file.


curl (7.15.3-1jab1) unstable; urgency=low

	* Changed libssl-dev version dependency to 0.9.7e-3sarge1
	* Bumped version number in an upgrade friendly way

 -- Jason Boxman <jasonb@edseek.com>  Mon, 15 May 2006 23:32:33 -0400

Now, we can proceed to prepare our modified package.


$ cd ../..
$ tar -xzf curl_7.15.3.orig.tar.gz
$ mv curl-7.15.3 curl-7.15.3.orig
$ dpkg-source -b curl-7.15.3.jab/ curl-7.15.3.orig/    
dpkg-source: warning: source directory `./curl-7.15.3.jab'
  is not <sourcepackage>-<upstreamversion> `curl-7.15.3'
dpkg-source: building curl in curl_7.15.3.orig.tar.gz
dpkg-source: building curl in curl_7.15.3-1jab1.diff.gz
dpkg-source: building curl in curl_7.15.3-1jab1.dsc

Now, let us give this backport another try!


$ sudo pbuilder build curl_7.15.3-1jab1.dsc 
 -> Attempting to parse the build-deps :
 pbuilder-satisfydepends,v 1.24 2006/03/31 00:14:31 dancer Exp $
 -> Considering  debhelper (>> 5)
   -> Trying debhelper
 -> Considering  autoconf
   -> Trying autoconf
 -> Considering  libtool
   -> Trying libtool
 -> Considering  automake1.7
   -> Trying automake1.7
 -> Considering  binutils (>= 2.14.90.0.7)
   -> Trying binutils
 -> Considering  libssl-dev (>= 0.9.7e-3sarge1)
   -> Trying libssl-dev
 -> Considering  libgnutls-dev
   -> Trying libgnutls-dev
 -> Considering  zlib1g-dev
   -> Trying zlib1g-dev
 -> Considering  stunnel
   -> Trying stunnel
 -> Considering  libkrb5-dev [!hurd-i386]
   -> Trying libkrb5-dev
 -> Considering  libidn11-dev
   -> Trying libidn11-dev
 -> Considering  groff-base
   -> Trying groff-base
 -> Considering  libdb4.2-dev
   -> Trying libdb4.2-dev
 -> Considering  libldap2-dev
   -> Trying libldap2-dev
 -> Installing  debhelper autoconf libtool automake1.7 binutils libssl-dev
 libgnutls-dev zlib1g-dev stunnel libkrb5-dev libidn11-dev groff-base
 libdb4.2-dev libldap2-dev

Success!

Much later, the package successfully builds, too.


dpkg-deb: building package `curl'
  in `../curl_7.15.3-1jab1_i386.deb'.
dpkg-deb: building package `libcurl3'
  in `../libcurl3_7.15.3-1jab1_i386.deb'.
...

4.3.2. Backporting Package Dependencies

Another approach is to simply backport each dependency that is required until you can successfully build your target package.

In order to obtain libssl-dev (>= 0.9.8-1), we need to download the source package for OpenSSL. By searching packages.debian.org we discover that the source package that provides the dependency we require is openssl.


$ apt-get source openssl
Reading Package Lists... Done
Building Dependency Tree... Done
Need to get 3306kB of source archives.
Get:1 http://http.us.debian.org testing/main openssl 0.9.8a-8 (dsc) [799B]
Get:2 http://http.us.debian.org testing/main openssl 0.9.8a-8 (tar) [3271kB]
Get:3 http://http.us.debian.org testing/main openssl 0.9.8a-8 (diff) [34.0kB]
Fetched 3306kB in 22s (145kB/s)
dpkg-source: extracting openssl in openssl-0.9.8a

Above, we obtain the source package we determined builds libssl-dev (>= 0.9.8-1). Now let's build it.


$ sudo pbuilder build --logfile /tmp/ssl.build openssl_0.9.8a-8.dsc

Frequently it's convenient to log the build somewhere, so we can refer back to all the output that is produced. The --logfile for pbuilder does precisely that.


dpkg-deb: building package `libssl-dev' in `../libssl-dev_0.9.8a-8_i386.deb'.

Much to our relief, libssl-dev builds without incident and the package we require has been built. Now, we can try building curl again.


$ sudo pbuilder build curl_7.15.3-1.dsc 
W: /.pbuilderrc does not exist
I: using fakeroot in build.
pbuilder-buildpackage/i386 $Id: pbuilder-buildpackage-funcs,
 v 1.28 2005/12/21 11:57:29 dancer Exp $
$ Id: pbuilder-buildpackage,v 1.122 2006/03/17 22:17:16 dancer Exp $

Current time: Sat May 20 02:24:38 EDT 2006
...
 -> Attempting to parse the build-deps :
 pbuilder-satisfydepends,v 1.24 2006/03/31 00:14:31 dancer Exp $
 -> Considering  debhelper (>> 5)
   -> Trying debhelper
 -> Considering  autoconf
   -> Trying autoconf
 -> Considering  libtool
   -> Trying libtool
 -> Considering  automake1.7
   -> Trying automake1.7
 -> Considering  binutils (>= 2.14.90.0.7)
   -> Trying binutils
 -> Considering  libssl-dev (>= 0.9.8-1)
   -> Trying libssl-dev
...
-> Finished parsing the build-deps

Success!

The above process must be repeated for every dependency that cannot be met. Remember the presence of /var/cache/pbuilder/result as a local apt-get source is what feeds the pbuilder build environment, allowing it to use the libssl-dev package that was just backported.