Revision / Modified: Jan. 17, 2001
Author: Tom Berger
Original documents:
http://www.mandrakeuser.org/docs/admin/aproc.html
http://www.mandrakeuser.org/docs/admin/aproc2.html
Did you ever had some program 'hanging' on you? Indeed 'hanging' so badly that it gobbled up all system resources, slowing down all operations to a crawl? And didn't you wish there were just a way to kill that damn thing? Were you ever interested in knowing how much system resources a program uses?
Well, you have come to right place, then :-). What are processes? Basically everything going on on your system: your web browser is a process, your web server - if you happen to run one - is also a process. A process boots your machine and another shuts it down. Every tiny script starts a process, albeit usually a very short one. "A process" says the textbook, "is an instance of a program in execution."
Processes can be started, stopped, killed, given priorities, scheduled. You do many of these actions already without knowing about it: when you open a program, you start a process, by closing the program, you kill the process. It's not some fancy Linux thing either, every modern operation system has them, because processes allow for multi-tasking: by keeping a unified list of what is going on, the operating system can run a schedule, thus providing its user with the illusion of being able to do many things at once.
Usually the operating system,i.e. the kernel, manages processes for you, you only need to interfere if you have special preferences (e.g. want to have a process to get more resources) or if a process goes awry and starts eating all the available system resources.
There are graphical interfaces for these programs, however, which you might find more convenient and easier to use: 'ksysguard' (package 'kdeadmin', menu: Applications - Monitoring - KDE System Guard), 'kpm' (package 'kdeutils', menu: Applications - Monitoring - Process Management) and 'gtop' (package 'gtop', menu: Applications - Monitoring - GTop). 'ksysguard' even allows you to watch and control processes on other machines.
Since all these tools use the same information from the '/proc' directory
(more on that later), they all display processes and their attributes in
similar fashion.
If you open them, you'll find a dynamic (apart from 'ps') column display with
a row for each process. Columns are:
PID: This is the Process IDentification number. To the system,
processes are not known by their name, but by a unique number. Each new
process gets a higher PID. The more processes you run and / or the longer your
machine runs, the higher those numbers get. That's normal.
User (also Login): The user who 'owns', i.e.
started this process. Users can only manipulate processes they own. Only
'root' can manipulate all processes.
Pri (also Nice or NI): The priority
of a process in relation to all other processes. Lower numbers indicate
higher priority. Standard priority is '0'.
Size (also VSZ or VmSize): The
total amount of virtual system memory a process uses. As
explained later on, that's by no means a fair indicator of how much
memory the process actually uses. Compare next entry.
Resident (also VmRSS or RSS): The
amount of physical system memory a process uses up. This is more accurate for
measuring how much system memory a process needs.
Stat (missing in 'ksysguard'): The status of a process. Sane
processes are either 'R'unning or 'S'leeping. It is normal that only one
process is displayed as 'running' at any given time while the others are
'sleeping' (i.e. waiting for a signal).
CPU (subdivided in User% and System%
by 'ksysguard'): Percentage of the processor time taken up by the process
since the last display refresh.
MEM: Percentage of physical system memory consumed by the
process.
Time (missing in 'ksysguard'): Total amount of processor time
the process has used since started.
Command (also Cmd): Command name of the
process.
Manipulating processes via the graphical utilities usually just involves 'highlighting' the process you want to handle and using the right mouse button click context menu. 'top' is controlled via shortcut keys, read man top for a list.
Notice that there's a standard key combination for killing graphical applications, <ALT> <CTRL> <ESC>. This will change the mouse pointer to a 'death' symbol. Click on an application window and that application will be killed. You still want to check if the processes of that application had been killed, too, though.
The father of all processes is 'init' (initializer). The pstree
command shows this quite impressively.
'init' manages the runlevels,
it controls which processes are started and stopped at which runlevels and it
is responsible for bringing the system to these runlevels. If you shutdown
your system, you essentially tell 'init' to bring it to runlevel 0. It's the
same if you type shutdown -h now or init 0.
Information on current processes is stored in the '/proc' file system. The '/proc' file system consists of kernel data changed in real-time. It is therefore not a 'real' file system like the others: the data contained take up no place on the hard-disk and you can't write to it. But you can extract and change the information contained therein (e.g. using the 'cat' command).
Listing '/proc', you'll find a lot of directories which names only consist
of numbers. These numbers are the 'process IDs' (PIDs). If you list such a
directory, you'll find files like 'cmdline', which contains the command line
the process was started with, or 'stat', which contains number codes on the
current process.
For example:
# cat /proc/1/cmdline
init
Which means: 'The process with the PID 1 was started with the command 'init''. 'init' will always be the first process and thus is the only process with a permanent PID. All other processes will have increasing PIDs, the maximum PID being 32768 (although the maximum number of processes on Linux is only limited by the amount of physical memory available).
The children of 'init' can have child processes themselves. If you start a
program by typing its name on a command line, the program will be a child of
the program which handles the command line. In most cases, this will be 'bash'
(the 'Bourne-Again-SHell').
One important thing to know is that child processes 'inherit' some assets of
their fathers. If you run a process with 'root' privileges, a child of this
process will also have these privileges.
Now you might ask: 'Since init runs with 'root' privileges, and it is the father of all processes, does that mean that all processes and programs on Linux run with root privileges?'
It's obvious they don't do that. Servers like the font server, web servers,
mail servers run on their own user account. The font server for example runs
on the 'xfs' account, the Apache web server on the 'apache' user account.
Servers which don't come with their own user account use the standard 'nobody'
user account. These are not 'real' accounts: nobody can login via these
so-called system accounts.
The reason for this is security: servers can be accessed from outside the
machine. If one of them contains a software flaw which allows attackers to
execute commands on your machine as the user who is running the server, it
would be a disaster if that user would be 'root'. Notice that this isn't a
system mechanism, but a standard feature of Linux servers. You should refrain
at all cost from using servers which do not include it.
Another mechanism is the 'login' command. Have a look at the output of 'pstree' again. You will see that every process and program you started after boot is a child of the 'login' process. 'login' switches the permissions of these child processes depending on how you log into the system. If you log in as 'root', all children will have 'root' privileges, if you log in as a user, the children will run with the permissions (the 'UserID', UID) of this user. The latter is preferable for the system's security and stability. If you need 'root' privileges, you can always log into the 'root' account from your user account by using 'su' or 'sudo'.
As you've already seen, processes can run with a UserID different from the user who started them. Servers are started by 'root' but drop this privilege as soon as possible and run on their own system account. In Unix-speak, one says they are running 'setuid account' (e.g. Apache runs 'setuid apache').
Of course, you can also do that the other way round by providing processes
started by a user root privileges.
The most prominent example is the graphical subsystem, 'X'. 'X' needs 'root'
privileges to run. However, the children of X do not run as 'root'. How
come?
'X' is started via a so-called wrapper, '/usr/X11/bin/Xwrapper'. If you look
at the permission bits of
this program, you will see it's 'setuid root':
-rws--x--x. This wrapper handles the permissions
part when starting X, so that X itself doesn't need to be 'setuid root'. It
runs as 'root' but its children don't inherit its UID.
Processes run either in user mode or in kernel mode. The operating system and its core services run in the privileged kernel mode, whereas 'normal' applications run in user mode, this way badly behaving user processes do not interfere with system operation. Some user processes however need access to privileged kernel functions. For this they switch to kernel mode, execute the needed function and switch back to user mode.
All user processes get the same share on the processor schedule. But you
can change that with the nice -n value process command.
Negative values increase the priority. They can only be applied by
'root', whereas positive values can be applied by any user.
nice -n value process starts a process with a given
priority, the command renice priority PID changes the
priority of a running process.
A program may open several processes if needed. This however isn't very
efficient: each process would need its own address (memory) space and
switching between these would cost a considerable amount of system resources.
This is where 'threads' come in. Threads are started by a process,
but they remain within the original address space used by the process.
Switching between threads consumes much less resources than between processes.
Applications which use this technique are called 'multi-threaded'.
Programming thread-enabled applications cleanly is very hard which justifies
the use of threads only in very complex pieces of software, like the Mozilla
browser.
As a system administrator, you only have to know about them that process monitors on Linux can not discriminate a thread from a process. Point in case is the Mozilla browser. If you start it, the process monitors will show multiple entries for it:
tom 11290 13.4 2.3 33080 21124 ? S 11:47 0:02 /usr/lib/mozilla/ tom 11296 0.0 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/ tom 11297 0.0 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/ tom 11298 0.0 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/ tom 11299 0.7 2.3 33080 21124 ? S 11:47 0:00 /usr/lib/mozilla/
This does not mean that the Mozilla browser has started five processes, but that's one process with four threads, all in the same address space. How can one tell? Well, you don't get any real proof, since the monitor doesn't know either, but there are hints like the same memory consumption numbers, the same command name and the same start time. Only the first entry is valid when looking at system resource usage.
For more on Linux process management, read the Linux Gazette article Processes on Linux and Windows NT by Glen Flower.
Process accounting enables you to keep detailed accounting information for the system resources used, their allocation among users, and system monitoring.
(Enabling Process Accounting on Linux HOWTO)
You will need to install the 'psacct' package from the first Mandrake Linux
CD which will in turn install a system
service of the same name. Documentation (and an amusing bit of Unix lore)
can then be found in info accounting.
Both 'Webmin' and 'Linuxconf' offer graphical modules to configure and
monitor process accounting.
Estimating the resource usage, especially the memory consumption of
processes is by far more complicated than it looks like at a first
glance.
How much resources a process needs, depends on many factors, most of them
varying. One is the amount of physical RAM in the system. If there is much
free RAM available, more caching will be performed and thus more memory
consumed. However this doesn't really count as resource usage, since this
cached memory is available in case some other process needs it. Up to 20% of
system memory can be used for buffering and caching. This dramatically
improves disk accesses, so you don't 'loose' anything when memory is assigned
to this task. Only unused memory is wasted ;-).
Graphical applications depend on certain collections of code libraries for displaying their interface (buttons, menus etc). Common collections are GTK+ (GIMP, GNOME), Qt (KDE), Tcl/Tk and others. If you use applications of many different collections, all these libraries have to be loaded into system memory, thus using more resources. An application written with Qt for KDE will use more system resources in GNOME than on KDE. If you start a second Qt application, however, it will use about as much resources as on KDE, since the first application already did load the needed interface libraries. If you close both applications, not all resources will be freed up again, instead the loaded libraries will be cached (if the amount of free RAM permits it).
The numbers for each process presented by process monitors tend to mislead and are easily misinterpreted. On the previous page you've already read about threads. But even if the process doesn't spawn threads, it is in most cases almost impossible to determine how much system memory an application really consumes.
Let me give a simple example:
According to the process table entry, 'bluefish' uses 4716 KB of real
memory and 3580 KB of shared memory. The system however only reports an
increase of 2404 KB general memory usage, of these being 400 KB more
in shared memory, 30 KB more in buffers and 600 KB more in cache.
Regardless of how you add or subtract these numbers, you will never get the
numbers from the process entry and the system numbers in sync.
I rather trust the system numbers, but even they can be misinterpreted quite easily:
42 processes: 41 sleeping, 1 running, 0 zombie, 0 stopped CPU states: 0.9% user, 1.3% system, 0.0% nice, 97.6% idle Mem: 257676K av, 252940K used, 4736K free, 202852K shrd, 7464K buff Swap: 130748K av, 256K used, 130492K free, 197620K cached
What usually confuses people here is the cache and buffer management of Linux. Let's have a look at it:
Mem: 257676K av, 252940K used, 4736K free, 202852K shrd, 7464K buff Swap: 130748K av, 256K used, 130492K free 197620K cached
'Mem' shows some 252 MB RAM installed. In fact it's 256 MB. What happened
to the remaining 4 MB? Simple: they are used for 'shadowing' the system's BIOS
and contain the GNU/Linux kernel.
The next fields seem to indicate that the system is on the brink of collapse:
almost all of the memory seems to be used up. Is that so? No, it isn't. Look
at the last entries of those lines. These tell you that about 200 MB of system
memory are used for caching and buffering. This memory is available for every
application which needs it.
The 'free' command line tool gives a more comprehensible overview:
$ free
total used free [...]
Mem: 257676 253624 4052 [...]
-/+ buffers/cache: 50360 207316
Swap: 130748 256 130492
Have a look at the third line:
-/+ buffers/cache: 50360 207316
displaying again the actual amount of memory in use by applications (50 MB) and free (202 MB). The rest is used for caching and buffering, i.e. to make your system faster.
To estimate the resource consumption of a process, it's easiest to run 'free' before, during and after running the process. The figures will vary because of the already mentioned reasons, so you should do this several times to get an average.
PAM (Pluggable Authentication Modules, man 8 pam) offers a
mechanism to set limits to resource usage, setting limits via shell mechanisms
('ulimit' in bash, 'limit' in csh etc) is deprecated.
You configure these limits as 'root' in '/etc/security/limits.conf'. Among
other things you can set the maximum number of processes per user or group,
process priorities, maximum CPU time and more.
Parameters and examples are supplied in the configuration file.
More information can be found in the appropriate chapter of the L-PAMSAG.
If a process behaves badly, the kernel sends it a certain signal (man
7 signal) to terminate it.
The most infamous signal is SIGNAL
11 (alias SIGSEGV, alias 'segmentation fault'). The process has tried to
access a memory segment which hasn't been allocated to it.
Reasons for a segfault might be a programming error, a wrong library version
or a hardware failure. The default action of a process when receiving a Signal
11, is to terminate itself and write the contents of the system memory to a
'core' file (in Unix slang, the process 'dumps the core'). This core file can
be useful to a programmer for debugging. Notice that Mandrake Linux by default
disables the creation of core files.
Things get a bit hairy however when the process is that bad that it ignores this signal and starts to hoard all the available processor time on the system, thus slowing down ('starving out') all the other processes. The kernel marks this process as 'uninterruptible' and its status in the process table will be switched from 'R' or 'S' to 'D'. If you see a process entry with a 'D' in its STATUS row, it's definitely time for you to do something.
Almost all signals can be handled by processes which means that they can also be ignored by them. There is however one signal which can't be ignored by any process and that's SIGNAL 9 (SIGKILL), the appropriately named 'kill signal'. If you send that signal to a process, you will quite literally kill it by taking away all its resources and removing it from the process table. You can then safely go on in your daily work. All unsaved data of that process will be lost, too, though.
Most system monitors offer you a possibility to kill a process via the right mouse click context menu (in console 'top', press the <k> key). There is one problem though: processes run with the permissions of the user who started them. A user can only kill processes he himself has started, only 'root' can kill all processes. Your system monitoring program will and should usually run under your user account which means you can't kill processes from other users or 'root' via it.
If a process started by root or another user misbehaves, you have to switch
to the 'root' account on a console using the su command. You can
then either kill the process by its process ID (PID) or by its name.
If you don't know the PID of the process yet, you can find out with
# ps aux --sort %cpu
which lists the most CPU hungry process last, the PID being the leftmost number. You can then go on to actually kill the process with the 'kill' command:
# kill -s 9 PID
You can also kill a process by name. Notice that this can be dangerous, since you either might accidentally kill processes you didn't intend to kill (or none at all). The command is
# killall -s 9 name
If you ever happen to come across a non-Linux Unix system, make sure to read the man page for 'killall' on that system. Some Unixes take this command literally which can get very ugly ...
A rather harmless type of a bad process is the so-called 'Zombie' (STATUS: Z, 'defunct'). If you've read the previous page, you know that processes can have 'children', i.e. spawn new processes. If the parent process is terminated normally, it will send a termination signal to all its children signals, too. If however the parent is terminated abnormally, it might not have gotten around to send its children the termination signal. Without their parent process these processes turn into 'living dead': they still appear in the process table, but they don't use any resources and there is no way of 'reaching' these processes. Unlike their namesakes, process zombies don't do any harm. They will vanish from the process table upon the next reboot.