Building for multiple Linux distributions

When distributing software for Linux, it is useful to be able to try it out on several different distributions. Complex programs with many dependencies can present particular problems. Building binary packages for several platforms may be necessary.

However, having one machine for each target Linux distribution is wasteful. There are simpler approaches. These include:

  1. Portable binaries. The apgcc component of the autopackage project allows completely portable linux binaries to be built for applications which do not link too heavily into the kernel or X-windows system. Typically, it is only necessary to install apgcc and then rebuild your software with "make CC=apgcc CXX=apg++".
    However, this approach is more tricky for Fortran, or for software with deep X or kernel dependencies.
  2. Virtualisation with QEMU. This is a technique which allows you to run several cirtual PCs with different operating systems within a real PC. The commercial VMware software will do this, but you can also use QEMU for free. This is ideal for testing, especially for graphical applications. The disadvantages are that a graphical display is used, so remote working over a slow link may be inconvenient. Performance is fairly good with the latest version.
  3. Change-root. This approach is very simple, gives best performance, and can be used remotely over a slow link. The target OS is installed in its own partition on the build machine. It need not be added to the boot loader however. Whenever that OS is required, the chroot command is used to start a shell using the root directory of that distribution instead of the current one. You can then use all the softare of the target distribution with the host kernel.

There follow some notes on using these approaches.

Using apgcc for Fortran

The following may work: 

setenv FC apgcc
setenv XLDFLAGS '-lfrtbegin -lg2c'

Using apgcc for C++

apg++ works great for compiling C++ objects, but if you use it for linking, the libm is linked dynamically, as is libgcc_s.

This can be avoided using "make CC=apgcc CXX=apg++ LD=apgcc", and linking using "-static -static-libgcc".

Your link command will also need to explicitly include the following libraries: "-lstdc++ -lm".

The resulting binary should run on any system which libc6 on it, which covers most Linux distributions back to the around the mesolithic period.

Using QEMU

To install QEMU, simply unpack the binary distribution from the QEMU website.

For good performance, you will want to install the kqemu accelerator. We will use the qemu binaries, and build kqemu for the host kernel. For this you must download kqemu source packages, in addition to installing the qemu binary package. Unpack and build kqemu using: 

tar xvfz ../kqemu-1.3.0pre8.tar.gz
cd kqemu-1.3.0pre8
./configure --disable-gcc-check
make

Then become root and 

make install
/sbin/modprobe kqemu major=0

Now if you run qemu as root, you should get good performance for common tasks. For userspace acceleration on recent Fedora, you need something like: 

# kqemu device (KDC)
KERNEL=="kqemu", MODE="0666"

at the bottom of /etc/udev/rules.d/50-udev.rules and reboot. If you did not add the modprobe command to your startup scripts, you will have to redo that.

To install an OS, download an ISO, say distro.iso (this needs to be a single image, so DVD preferable), and use: 

qemu-img create distro.img 4G
qemu -hda distro.img -cdrom distro.iso -boot d -m 256 -kernel-kqemu

After installation, you can use the distribution with: qemu distro.img -m 256 -kernel-kqemu

Using QEMU to build chroot environments

It would be nice to be able to install a target OS using QEMU so that you don't need to reboot the host computer. But you need to solve the problem of getting the installed OS out of the image file. This should be possible using the loop device, but I haven't made it work.

A workaround is to tar the target OS from within QEMU and then use sftp to copy the tar file out to the host machine. You probably want to exclude the /proc, /dev and parts of the /sys directories.

Using chroot environments

Using recent kernels, the /proc, /sys and /dev directories are important to the interaction of the kernel and the OS. When creating a chroot directory heirarchy from an installed image (either native or in QEMU), it is simplest to tar up all of the directory tree except for these directories.

After untarring the tree in a the host filesystem, create empty /proc, /sys and /dev directories. Then link these to the corresponding directories of the host system before issuing the chroot command, e.g. 

mount --bind /proc /home/cowtan/suse10/proc
mount --bind /dev /home/cowtan/suse10/dev
mount --bind /sys /home/cowtan/suse10/sys
/usr/sbin/chroot /home/cowtan/suse10/
Back to my homepage.
Kevin Cowtan: