Compiling

Revision / Modified: Apr. 21, 2002
Author: Tom Berger

Original documents:
http://www.mandrakeuser.org/docs/basics/bsource.html http://www.mandrakeuser.org/docs/basics/bsource2.html http://www.mandrakeuser.org/docs/basics/bsource3.html

Compiling Basics

A program is written using a programming language. Some languages, the so-called 'script languages', allow you to run written programs right away. Popular examples for these kind of languages are Perl, Python and Tcl/Tk. They come with an interpreter which serves as an translator between the program and the system.

Other languages like C or C++ require the written program to be compiled to a 'binary' before it can be run.

Each approach has its pros and cons: Interpreted programs require less time to develop, whereas binary programs do not need an interpreter and thus run faster, and are usually better suited for complex tasks.

Compiling translates the human-readable text, the 'source', into a machine-readable file, the 'binary'. So the first thing you need for compiling is a compiler. In Linux, that's 'gcc' for programs written in C (the majority), and 'g++' (aka 'gcc-c++') for programs written in C++.
Complex programs can consist of large number of source files. In order to make the compilation easier, compilation instructions are often listed in a file, the 'Makefile'. To make use of this file, you need the program 'make'. 'make' reads through ('parses') the Makefile and provides the compiler with the appropriate options. You can change these options (like for instance where the compiled files will be installed) by editing the 'Makefile'.

When writing programs, the programmer doesn't write the whole program. He or she relies on 'libraries' which supply a set of commonly used functions. The programmer just sets a 'link' in the source file referring to a certain library or a function in that library. The compiled binary will need this library to run.
Now, if you compile the program, the compiler will need access to some of the source files of each library the program depends on. These files are called 'header files'. So you will need these, too.
Since you need these header files only for compiling, it might be sensible to split RPM-packaged libraries into two: one package which contains the files needed to run a program and one package which contains the files needed to compile a program. And this is indeed what is now commonplace in distributions: one package contains the library itself (the 'runtime version') and a second package, marked with '-devel-' (for 'development'), contains the header files.
So, you will need the packages which contain the header files of the libraries the program depends on ,too.

You might ask yourself how you are supposed to know which libraries a program needs. The 'simplest' way would be trying to compile the source. The compilation process will bail out if it can't find a needed library.
The second possibility is having a look at the Makefile itself. At the beginning there is a line starting with LIBS =, which lists all the libraries needed. Notice that they are listed in some sort of shorthand: '-lX11' for example is shorthand for the filename 'libX11.so'. Do a locate libX11.so to find out if this file is installed on your system.
A third mechanism is the 'configure' script, which a programmer can produce using the 'autoconf' program. This script, when run, checks your system if all the needed libraries and functions are available on your system. If not, it will fail and tell you what's wrong.

Unpacking Archives

Source is usually distributed in compressed archives. Which makes sense, since it usually contains of more than one file and compression works especially good on text files.

In UNIX, these archives are created and expanded by using two different programs: one that collects all file in an archive ('tar') and one which does the compression. By convention, gzip compressed archives have the ending 'tar.gz' or 'tgz', whereas bzip2 compressed archives end on 'tar.bz2' or 'tbz2'. These archives are colloquially called 'tarballs'.

To expand archives, you can either use some graphical archive manager like KDE's 'Archiver', GNOME's 'guiTAR' or TkZip. In KDE and GNOME, they should be started automatically as soon as you (double-)click on an archive file in the respective file manager.
On the command line, you use:

tar xzf archive for extracting gzip-compressed tar file archives, and

tar xjf archive for tar files compressed with bzip2 (older version of tar might require 'y' instead of 'j').
The first option ('x') means 'extract'. For creating archives, you'd substitute it with 'c', to merely list the contents with 't'), the second denotes the compression utility and the 'f' tells a 'tar' to expect a filename. One other interesting option is 'C' which tells 'tar' to switch to another directory before extracting the archive:

tar xzfC archive dir

"Pure" 'bzip2' or even 'zip' archives are quite rare in the Linux world. Unpack such archives with bunzip2 archive or unzip archive (the latter works on self-extracting '.exe' archives, too).

Important Files to Read

Most source archives contain files called README and INSTALL. They feature comments by the programmer on the program, how to compile it, things to watch out for and more. It goes without saying that you should read these files ;-), except if the INSTALL file begins with 'These are generic instructions', in this case the programmer couldn't be bothered to replace the file generated by 'autoconf' by something specific and helpful.
Like every other file in the archive you can open them with every program that is capable of displaying text files.

Needed Packages

Here's a checklist of packages you will need for compiling. Check with rpm -q name if these are installed.
If you have a 1-CD version of ML, some of these might not be on that CD; in this case get them via rpmfind.net.

Compiler

System

Popular Libraries

There are libraries which are used very often by applications. Having their development counterparts installed should get you through most compilations without errors due to missing libraries. These development libraries are the most popular with programmers. If you have them installed, you should get through most compilations without errors due to missing libraries.

If you are compiling programs for GNOME or KDE, you will need the '-devel-' versions of their libraries. For KDE, you will also need 'libqt-devel'.

There are two things to note about package names for libraries:

  1. Older Mandrake Linux library packages might not have the 'lib' prefix, e.g. 'libgtk+-devel' was 'gtk+-devel'.
  2. Some library packages might have a version number in their name, e.g. the actual and current name of 'libpng-devel' is 'libpng3-devel'. This isn't important, I just mention it here to avoid confusion ;-).

Running 'configure'

Nowadays most source packages come with a 'configure' script, which checks if the program can be compiled on the current system.

./configure --help

executed in the source directory, gives you a summary of available options to the 'configure' command. Don't get confused, default values should be OK in most cases.
There is one option, though, you should have a closer look at and that's '--prefix=[DIR]', very close to the beginning of the options list. This is where the compiled programs and libraries will be installed. The value is often set to '[/usr]' or '[/usr/local]'. Both values are alright, though I prefer to install self-compiled programs in the '/usr/local/' hierarchy because that helps keeping track of things.
However some KDE programs assume you have a directory '/opt/kde' and have this as their default installation directory. Mandrake Linux might have such a directory, but since KDE files are installed in the '/usr' hierarchy, you have to change that value by running 'configure' like this:

./configure --prefix=/usr

You might have wondered already why I wrote './configure' instead of 'configure'. This is because the freshly created source directory usually is not part of your shell path. So if you type configure, the shell will say:

bash: configure: command not found

You have to tell the shell that you want to execute a command that is in the current working directory. And this is done by adding the prefix './' to the command.
More about path settings in Linux.

Notice that it is neither necessary nor recommended to run 'configure' scripts as 'root'.

What's Missing?

Now 'configure' runs and checks your system. The process puts its output on the console and sends it to a new file called 'config.log'. It automatically stops if it encounters an error that would prevent the program from being successfully compiled.
It displays an error message and writes it to 'config.log', e.g.:

/usr/bin/ld: cannot open -lXt: No such file or directory
collect2: ld returned 1 exit status

Looks like there is something missing, but what is '-lXt'? That's a convention in programming: 'l' stands for 'lib...so'. So what's needed is the file 'libXt.so'.
Chances are that it is in some -devel-.rpm you haven't installed. If you are running Mandrake Linux 7 or later, use the 'Software Manager' (aka 'RpmDrake') to search for a package containing this file ('Tree - See available packages' and then 'Search - File') or use urpmf file on the command line.
On older releases, go to the directory on the CD which contains the RPMs (/mnt/cdrom/Mandrake/RPMS) and run this command:

for i in *.rpm ; do rpm -qpli $i | grep file && echo $i ; done

(Replace file with the name of the file you are looking for).
You will see that it belongs to 'XFree86-devel'. Install this RPM using your preferred graphical RPM application, 'urpmi' or via

su -c 'rpm-i XFree86-devel*'

Now delete the file 'config.cache' in the source directory and run 'configure' again. Repeat until 'configure' exits without errors.

If you don't find an RPM which contains the file you need (very unlikely) and the included documentation doesn't help you either, try to contact the author of the software. He or she knows best. But make sure you've checked everything (at least) twice!

Running 'make'

If 'configure' was successful, it's now time for actually compiling the source with

make

There shouldn't occur any errors unless the 'configure' script was faulty (well, accidents happen).
Though 'make's error messages can be somewhat more obscure than those of 'configure', the procedure in such cases is very much like the one outlined above.
If something goes wrong, it is usually something like

file:line: file: No such file or directory

or

file1: in file | function name:
file2:line: undefined reference to function

In the first case, install the RPM which contains the missing file. An error containing a function name usually means that the program needs a newer or older version of a library than the one you have installed. See if you can find an updated or older version of this library and install it.

Always run

make clean

before trying to compile again.

Let's assume everything has worked out fine. Installation time! You have to be root to do this:

su -c 'make install'

Ready. You might want to check now if the program runs at all ;-). Make sure you put the binary in a directory which is part of your path, otherwise create a 'symlink' with ln -s source destinationto '/usr/bin' or add the directory to your path.

Uninstalling

Many people do not know this, but often it is quite easy to uninstall files that have been installed via make install. You just need the Makefile and then - being in the same directory as the Makefile - type:

su -c 'make uninstall'

provided the author defined this target in his Makefile (many do).

If you regularly install programs from source, you might want to have a look at CheckInstall, which builds simple RPMs from compiled programs and then installs those RPMs. This makes removing those programs much easier (you can use rpm -e name and also prevents dependency problems when you later install RPMs which depend on those files.
(Note to Mandrake Linux 8.2 users: do not use the CheckInstall RPM from your CD, it won't work. Get the RPM from the website).

Applying Patches

A patch or diff file is a specially formatted text file containing instructions for the 'patch' program what and where to change source files.

A 'patch' command looks like this:

patch -pnumber <patch_file

The tricky part is the number after the '-p' option. This defines to what extend the file path in the patch file should be applied to your system:
'-p0' tries to apply the full path given in the patch file, '-p1' removes the first slash etc
Not specifying '-p' at all strips the whole path, which is perfectly OK if the patch only applies to file(s) all residing in the same directory.

What usually works is putting the patch file in the parent directory to the directory of the source files to be patched, and running

patch -p1 <patch_file

If it doesn't, try other numbers to the '-p' option.

Compiling FAQ

(You've already read the first two pages of this article, haven't you? ;-))

Missing Or Unusual Build Files

The source directory doesn't contain a 'configure' file, what now?

In this case you have to deal with the 'Makefile' on your own.
Only some of the first few lines starting with words in capitals are of interest and may have to be changed. Makefiles may not contain all of these lines.

Now just run make.

What if the source directory contains an 'Imakefile', 'GNUmakefile' or 'Makefile.cvs' instead of an ordinary Makefile?

'Imake' is a predecessor to 'configure'. Running the command

xmkmf -a

will produce a Makefile from the Imakefile. This command is part of the XFree86-devel package. Then proceed as above.

There's no difference between GNUmakefiles and Makefiles, 'make' will handle both.

If you've got a 'Makefile.cvs', run

make -f Makefile.cvs'

This will result in a 'configure' file.

What if the source directory contains no build files at all?

First have a look for them in the subdirectories of the source directory. Then check if there are any other executable files:

find . -type f -perm -700

will list all executable files in the current directory and its subdirectories.

If you can't find anything, it is likely that you are supposed to run the compiler 'bare' on the source files. This is all the more likely if there's just a single *.c file. To do this:

gcc -o new_name file.c

'-o new_name' defines the name of the resulting binary. Omit the '-o' option if the directory contains more than one *.c (*.C, *.cc, *.cxx) file. In this case, the resulting binary will get the standard name 'a.out', you then might want to rename it to something sensible.

If you need to define extra include or library directories, or libraries, you have to do so by providing them as command line options to 'gcc' directly, e.g.

gcc -o new_name file.c -L/path/dir -llib -I/path/dir

Build File Problems

How come 'configure' complains about a missing library / header file although this library / header file is installed?

There are at least three scenarios which might lead to this error:

  1. 'configure' doesn't find the directory which contains the library.
    this case, you have to add the full path of the directory which contains the library using the '--with-extra-libs=[DIR]' option to 'configure'. The same goes for header files ('--with-extra-includes=[DIR]').

  2. 'configure' finds the wrong version of a library.
    Currently, a prominent example of this behavior are applications to be compiled with Qt2. 'configure' might fail with a misleading error message, if the installed minor version of Qt2 is too old (e.g. 2.2.1 when 2.2.2 or better is needed). In this case, you have to upgrade to a later minor version.
    Another problem frequently occurs if Qt1 and Qt2 (and Qt3 ...) are installed. By default, 'configure' looks for Qt in '/usr/lib/qt', but on ML, that's the directory where Qt1 is installed. Qt2 is in '/usr/lib/qt2', Qt3 in '/usr/lib/qt3'. In this case, you'd have to run 'configure' like this:
    ./configure --with-qt-dir=/usr/lib/qt2.

  3. Typo in 'configure' script.
    Have a close look at the error messages in 'config.log', especially for capitalization errors in the library or header file names. This is very rare (had that once or twice in some years), but it can really drive you nuts if you don't think of this possibility, believe me ... ;-).

Why does my self-compiled binary lacks some features it should have?

It is up to the programmer to decide upon which missing libraries 'configure' or 'make' should abort with an error message. Thus a successful compilation doesn't guarantee that the binary is feature complete. Study the 'config.log' file closely to find out which libraries weren't found, install these, and run the process again.

Building Problems

I've installed a missing library, but 'configure' or 'make' still won't find it. Why?

'configure' and 'make' results are cached. Do a

make distclean

to clean up the source directory. Only by this you will ensure that they check the system anew.
If the Makefile doesn't offer a 'distclean' target, do this

make clean && rm config.cache

Another source for this kind of problem might be if the library has been installed to a directory which isn't listed in '/etc/ld.so.conf'. This file lists all the directories where the Linux Loader/Linker 'ld' should look for libraries in addition to the standard locations, '/usr/lib' and '/lib'.
If for example the newly installed library is in '/usr/local/lib' and that directory isn't mentioned in '/etc/ld.so.conf', this library isn't available to the system.
To fix this, add this directory to '/etc/ld.so.conf' and run (as 'root')

ldconfig

Now clean up the source directory and try again.

How do I fix library versions conflicts when all the needed libraries are installed?

In most cases the build process doesn't look for a specific minor version of a library. It relies on properly set up symlinks in '/usr/lib'.
Take for example the library listing of 'libungif.so' in '/usr/lib' (abridged):

lrwxrwxrwx /usr/lib/libungif.so -> libungif.so.4.1.0*
lrwxrwxrwx /usr/lib/libungif.so.3 -> libungif.so.3.1.0*
-rwxr-xr-x /usr/lib/libungif.so.3.1.0*
lrwxrwxrwx /usr/lib/libungif.so.4 -> libungif.so.4.1.0*
-rwxr-xr-x /usr/lib/libungif.so.4.1.0*

There are two versions of this library installed: libungif.so.4.1.0 and libungif.so.3.1.0 (marked with *). Plus there are three 'symlinks' (entries starting with an 'l'): libungif.so, libungif.so.3 and libungif.so.4.

'Symlinks' aren't 'real' files, they are just pointers to other files.
The build process would look in '/usr/lib' for 'libungif.so.3' and be pointed by the symlink to the actually installed minor version of this library, here 3.1.0.

Now assume the programmer had been a bit unspecific (accidents happen) and the build process would only look for 'libungif.so'. In this case, it would be directed to libungif.so.4.1.0, a different major version of the same library. Major versions are often incompatible, so if the binary relies on major version 3, but the build process finds major version 4 instead, 'make' or 'configure' will terminate with an error, most likely complaining about a 'missing library' or an 'undefined reference'.
Likewise, if a symlink with the name of a major version pointing to the actually installed library is missing, the build process will fail, too.

For the first problem, a quick and dirty solution would be resetting the symlink for the build process

ln -sf /usr/lib/libungif.so.3.1.0 /usr/lib/libungif.so

If you encounter the second problem, create the needed symlink with the 'ln' command. The first file argument is the existing file, the second the name of symlink you wish to be created for this file.

You should be careful when doing this, since it might break other programs. If possible, check back with the programmer, they know best.


Legal: This text is covered by the GNU Free Documentation License. Standard disclaimers of warranty apply. Copyright LSTB (Tom Berger) and Mandrakesoft 1999-2002.