Revision / Modified: Apr. 21, 2002
Author: Tom Berger
Original documents:
http://www.mandrakeuser.org/docs/basics/bsource.html
http://www.mandrakeuser.org/docs/basics/bsource2.html
http://www.mandrakeuser.org/docs/basics/bsource3.html
A program is written using a programming language. Some languages, the so-called 'script languages', allow you to run written programs right away. Popular examples for these kind of languages are Perl, Python and Tcl/Tk. They come with an interpreter which serves as an translator between the program and the system.
Other languages like C or C++ require the written program to be compiled to a 'binary' before it can be run.
Each approach has its pros and cons: Interpreted programs require less time to develop, whereas binary programs do not need an interpreter and thus run faster, and are usually better suited for complex tasks.
Compiling translates the human-readable text, the 'source', into a
machine-readable file, the 'binary'. So the first thing you need for compiling
is a compiler. In Linux, that's 'gcc' for programs written in C (the
majority), and 'g++' (aka 'gcc-c++') for programs written in C++.
Complex programs can consist of large number of source files. In order to make
the compilation easier, compilation instructions are often listed in a file,
the 'Makefile'. To make use of this file, you need the program 'make'. 'make'
reads through ('parses') the Makefile and provides the compiler with the
appropriate options. You can change these options (like for instance where the
compiled files will be installed) by editing the 'Makefile'.
When writing programs, the programmer doesn't write the whole program. He
or she relies on 'libraries' which supply a set of commonly used functions.
The programmer just sets a 'link' in the source file referring to a certain
library or a function in that library. The compiled binary will need this
library to run.
Now, if you compile the program, the compiler will need access to some of the
source files of each library the program depends on. These files are
called 'header files'. So you will need these, too.
Since you need these header files only for compiling, it might be sensible to
split RPM-packaged libraries into two: one package which contains the files
needed to run a program and one package which contains the files needed to
compile a program. And this is indeed what is now commonplace in
distributions: one package contains the library itself (the 'runtime version')
and a second package, marked with '-devel-' (for 'development'), contains the
header files.
So, you will need the packages which contain the header files of the libraries
the program depends on ,too.
You might ask yourself how you are supposed to know which libraries a
program needs. The 'simplest' way would be trying to compile the source. The
compilation process will bail out if it can't find a needed library.
The second possibility is having a look at the Makefile itself. At the
beginning there is a line starting with LIBS =, which lists all
the libraries needed. Notice that they are listed in some sort of shorthand:
'-lX11' for example is shorthand for the filename
'libX11.so'. Do a locate libX11.so to find out if this
file is installed on your system.
A third mechanism is the 'configure' script, which a
programmer can produce using the 'autoconf' program. This script, when run,
checks your system if all the needed libraries and functions are available on
your system. If not, it will fail and tell you what's wrong.
Source is usually distributed in compressed archives. Which makes sense, since it usually contains of more than one file and compression works especially good on text files.
In UNIX, these archives are created and expanded by using two different programs: one that collects all file in an archive ('tar') and one which does the compression. By convention, gzip compressed archives have the ending 'tar.gz' or 'tgz', whereas bzip2 compressed archives end on 'tar.bz2' or 'tbz2'. These archives are colloquially called 'tarballs'.
To expand archives, you can either use some graphical archive manager like
KDE's 'Archiver', GNOME's 'guiTAR' or TkZip. In KDE and
GNOME, they should be started automatically as soon as you (double-)click on
an archive file in the respective file manager.
On the command line, you use:
tar xzf archive for extracting gzip-compressed tar file archives, and
tar xjf archive for tar files compressed with bzip2
(older version of tar might require 'y' instead of 'j').
The first option ('x') means 'extract'. For creating archives, you'd
substitute it with 'c', to merely list the contents with 't'), the second
denotes the compression utility and the 'f' tells a 'tar' to expect a
filename. One other interesting option is 'C' which tells 'tar' to switch to
another directory before extracting the archive:
tar xzfC archive dir
"Pure" 'bzip2' or even 'zip' archives are quite rare in the Linux world. Unpack such archives with bunzip2 archive or unzip archive (the latter works on self-extracting '.exe' archives, too).
Most source archives contain files called README and INSTALL. They feature
comments by the programmer on the program, how to compile it, things to watch
out for and more. It goes without saying that you should read these files ;-),
except if the INSTALL file begins with 'These are generic instructions', in
this case the programmer couldn't be bothered to replace the file generated by
'autoconf' by something specific and helpful.
Like every other file in the archive you can open them with every program that
is capable of displaying text files.
Here's a checklist of packages you will need for compiling. Check
with rpm -q name if these are installed.
If you have a 1-CD version of ML, some of these might not be on that CD; in
this case get them via rpmfind.net.
There are libraries which are used very often by applications. Having their development counterparts installed should get you through most compilations without errors due to missing libraries. These development libraries are the most popular with programmers. If you have them installed, you should get through most compilations without errors due to missing libraries.
If you are compiling programs for GNOME or KDE, you will need the '-devel-' versions of their libraries. For KDE, you will also need 'libqt-devel'.
There are two things to note about package names for libraries:
Nowadays most source packages come with a 'configure' script, which checks if the program can be compiled on the current system.
./configure --help
executed in the source directory, gives you a summary of available options
to the 'configure' command. Don't get confused, default values should be OK in
most cases.
There is one option, though, you should have a closer look at and that's
'--prefix=[DIR]', very close to the beginning of the options list.
This is where the compiled programs and libraries will be installed. The value
is often set to '[/usr]' or '[/usr/local]'. Both values are alright, though I
prefer to install self-compiled programs in the '/usr/local/' hierarchy
because that helps keeping track of things.
However some KDE programs assume you have a directory '/opt/kde' and have this
as their default installation directory. Mandrake Linux might have such a
directory, but since KDE files are installed in the '/usr' hierarchy, you
have to change that value by running 'configure' like this:
./configure --prefix=/usr
You might have wondered already why I wrote './configure' instead of 'configure'. This is because the freshly created source directory usually is not part of your shell path. So if you type configure, the shell will say:
bash: configure: command not found
You have to tell the shell that you want to execute a command that is in
the current working directory. And this is done by adding the prefix './' to
the command.
More about path settings in
Linux.
Notice that it is neither necessary nor recommended to run 'configure' scripts as 'root'.
Now 'configure' runs and checks your system. The process puts its output on
the console and sends it to a new file called 'config.log'. It automatically
stops if it encounters an error that would prevent the program from being
successfully compiled.
It displays an error message and writes it to 'config.log', e.g.:
/usr/bin/ld: cannot open -lXt: No such file or directory
collect2: ld returned 1 exit status
Looks like there is something missing, but what is '-lXt'? That's a
convention in programming: 'l' stands for 'lib...so'. So what's needed is the
file 'libXt.so'.
Chances are that it is in some -devel-.rpm you haven't installed. If you are
running Mandrake Linux 7 or later, use the 'Software Manager' (aka 'RpmDrake')
to search for a package containing this file ('Tree - See available packages'
and then 'Search - File') or use urpmf file on the
command line.
On older releases, go to the directory on the CD which
contains the RPMs (/mnt/cdrom/Mandrake/RPMS) and run this command:
for i in *.rpm ; do rpm -qpli $i | grep file && echo $i ; done
(Replace file with the name of the file you are looking for).
You will see that it belongs to 'XFree86-devel'. Install this RPM using your
preferred graphical RPM application, 'urpmi' or via
su -c 'rpm-i XFree86-devel*'
Now delete the file 'config.cache' in the source directory and run 'configure' again. Repeat until 'configure' exits without errors.
If you don't find an RPM which contains the file you need (very unlikely) and the included documentation doesn't help you either, try to contact the author of the software. He or she knows best. But make sure you've checked everything (at least) twice!
If 'configure' was successful, it's now time for actually compiling the source with
make
There shouldn't occur any errors unless the 'configure' script
was faulty (well, accidents happen).
Though 'make's error messages can be somewhat more obscure than those of
'configure', the procedure in such cases is very much like the one outlined
above.
If something goes wrong, it is usually something like
file:line: file: No such file or directory
or
file1: in file | function name:In the first case, install the RPM which contains the missing file. An error containing a function name usually means that the program needs a newer or older version of a library than the one you have installed. See if you can find an updated or older version of this library and install it.
Always run
make clean
before trying to compile again.
Let's assume everything has worked out fine. Installation time! You have to be root to do this:
su -c 'make install'
Ready. You might want to check now if the program runs at all ;-). Make sure you put the binary in a directory which is part of your path, otherwise create a 'symlink' with ln -s source destinationto '/usr/bin' or add the directory to your path.
su -c 'make uninstall'
provided the author defined this target in his Makefile (many do).
If you regularly install programs from source, you might want to have a look
at CheckInstall,
which builds simple RPMs from compiled programs and then installs those RPMs.
This makes removing those programs much easier (you can use rpm -e
name and also prevents dependency problems when you later
install RPMs which depend on those files.
(Note to Mandrake Linux 8.2
users: do not use the CheckInstall RPM from your CD, it won't work. Get the RPM
from the website).
A patch or diff file is a specially formatted text file containing
instructions for the 'patch' program what and where to change source
files.
A 'patch' command looks like this:
patch -pnumber <patch_file
The tricky part is the number after the '-p' option. This defines to what
extend the file path in the patch file should be applied to your system:
'-p0' tries to apply the full path given in the patch file,
'-p1' removes the first slash etc
Not specifying '-p' at all strips the whole path, which is
perfectly OK if the patch only applies to file(s) all residing in the
same directory.
What usually works is putting the patch file in the parent directory to the directory of the source files to be patched, and running
patch -p1 <patch_file
If it doesn't, try other numbers to the '-p' option.
(You've already read the first two pages of this article, haven't you? ;-))
In this case you have to deal with the 'Makefile' on your own.
Only some of the first few lines starting with words in capitals are of
interest and may have to be changed. Makefiles may not contain all of these
lines.
CC = compiler tells 'make' which compiler to use. GCC
should work in most cases. C++ programs may require G++ instead.INCDIR = -I/path/dir1 -I/path/dir2 etc tells 'make' in
which directories to look for header files. Check if these comply with your
setup.LIBDIR = -L/path/dir1 -L/path/dir2 etc tells 'make' in
which directories to look for library files. Check if correct.LIBS = -llib1 -llib2 -llib3 etc tells 'make' which
libraries are needed. Remember: '-llib' is short for 'liblib.so'. Use
locate liblib.so to check if the needed libraries are
installed. Sometimes this variable goes by the name LFLAGS.Now just run make.
'Imake' is a predecessor to 'configure'. Running the command
xmkmf -a
will produce a Makefile from the Imakefile. This command is part of the XFree86-devel package. Then proceed as above.
There's no difference between GNUmakefiles and Makefiles, 'make' will handle both.
If you've got a 'Makefile.cvs', run
make -f Makefile.cvs'
This will result in a 'configure' file.
First have a look for them in the subdirectories of the source directory. Then check if there are any other executable files:
find . -type f -perm -700
will list all executable files in the current directory and its subdirectories.
If you can't find anything, it is likely that you are supposed to run the compiler 'bare' on the source files. This is all the more likely if there's just a single *.c file. To do this:
gcc -o new_name file.c
'-o new_name' defines the name of the resulting binary. Omit the '-o' option if the directory contains more than one *.c (*.C, *.cc, *.cxx) file. In this case, the resulting binary will get the standard name 'a.out', you then might want to rename it to something sensible.
If you need to define extra include or library directories, or libraries, you have to do so by providing them as command line options to 'gcc' directly, e.g.
gcc -o new_name file.c -L/path/dir -llib -I/path/dir
There are at least three scenarios which might lead to this error:
'configure' doesn't find the directory which contains the library.
this case, you have to add the full path of the directory which contains the
library using the '--with-extra-libs=[DIR]' option to 'configure'.
The same goes for header files ('--with-extra-includes=[DIR]').
'configure' finds the wrong version of a library.
Currently, a prominent example of this behavior are applications to be
compiled with Qt2. 'configure' might fail with a misleading error message, if
the installed minor version of Qt2 is too old (e.g. 2.2.1 when 2.2.2
or better is needed). In this case, you have to upgrade to a later minor
version.
Another problem frequently occurs if Qt1 and Qt2 (and Qt3 ...) are
installed. By default, 'configure' looks for Qt in '/usr/lib/qt', but on ML,
that's the directory where Qt1 is installed. Qt2 is in '/usr/lib/qt2', Qt3 in
'/usr/lib/qt3'. In this case, you'd have to run 'configure' like this:
./configure --with-qt-dir=/usr/lib/qt2.
Typo in 'configure' script.
Have a close look at the error messages in 'config.log', especially for
capitalization errors in the library or header file names. This is very rare
(had that once or twice in some years), but it can really drive you nuts if
you don't think of this possibility, believe me ... ;-).
It is up to the programmer to decide upon which missing libraries 'configure' or 'make' should abort with an error message. Thus a successful compilation doesn't guarantee that the binary is feature complete. Study the 'config.log' file closely to find out which libraries weren't found, install these, and run the process again.
'configure' and 'make' results are cached. Do a
make distclean
to clean up the source directory. Only by this you will ensure that they
check the system anew.
If the Makefile doesn't offer a 'distclean' target, do this
make clean && rm config.cache
Another source for this kind of problem might be if the library has been
installed to a directory which isn't listed in '/etc/ld.so.conf'. This file
lists all the directories where the Linux Loader/Linker 'ld' should look for
libraries in addition to the standard locations, '/usr/lib' and '/lib'.
If for example the newly installed library is in '/usr/local/lib' and that
directory isn't mentioned in '/etc/ld.so.conf', this library isn't available
to the system.
To fix this, add this directory to '/etc/ld.so.conf' and run (as 'root')
ldconfig
Now clean up the source directory and try again.
In most cases the build process doesn't look for a specific minor version
of a library. It relies on properly set up symlinks in '/usr/lib'.
Take for example the library listing of 'libungif.so' in '/usr/lib'
(abridged):
lrwxrwxrwx /usr/lib/libungif.so -> libungif.so.4.1.0*
lrwxrwxrwx /usr/lib/libungif.so.3 -> libungif.so.3.1.0*
-rwxr-xr-x /usr/lib/libungif.so.3.1.0*
lrwxrwxrwx /usr/lib/libungif.so.4 -> libungif.so.4.1.0*
-rwxr-xr-x /usr/lib/libungif.so.4.1.0*
There are two versions of this library installed: libungif.so.4.1.0 and
libungif.so.3.1.0 (marked with *). Plus there are three 'symlinks' (entries
starting with an 'l'): libungif.so, libungif.so.3 and libungif.so.4.
'Symlinks' aren't 'real' files, they are just pointers to other
files.
The build process would look in '/usr/lib' for 'libungif.so.3' and be pointed
by the symlink to the actually installed minor version of this library, here
3.1.0.
Now assume the programmer had been a bit unspecific (accidents happen) and
the build process would only look for 'libungif.so'. In this case, it would be
directed to libungif.so.4.1.0, a different major version of the same library.
Major versions are often incompatible, so if the binary relies on major
version 3, but the build process finds major version 4 instead, 'make' or
'configure' will terminate with an error, most likely complaining about a
'missing library' or an 'undefined reference'.
Likewise, if a symlink with the name of a major version pointing to the
actually installed library is missing, the build process will fail, too.
For the first problem, a quick and dirty solution would be resetting the symlink for the build process
ln -sf /usr/lib/libungif.so.3.1.0 /usr/lib/libungif.so
If you encounter the second problem, create the needed symlink with the 'ln' command. The first file argument is the existing file, the second the name of symlink you wish to be created for this file.
You should be careful when doing this, since it might break other programs. If possible, check back with the programmer, they know best.