Linux Process Management: Data Structures and System Calls
Linux Process Management: Basic Data Structures and System Calls (Chapter 3)
Fundamental Process Concepts (Q1-Q4)
What is a process?
A process is a program in execution.
What are lightweight processes in Linux?
Lightweight processes are processes which offer better support for multithreaded applications.
Explain what is meant by a multithreaded application. Describe how multithreaded applications are implemented in Linux. Give three examples of POSIX-compliant pthread libraries.
A straightforward way to implement multithreaded applications is to associate a lightweight process with each thread.
In this way, the threads can access the same set of application data structures by simply sharing the same memory address space, the same set of open files, and so on. At the same time, each thread can be scheduled independently by the kernel so that one may sleep while another remains runnable.
Examples of POSIX-compliant pthread libraries that use Linux’s lightweight processes are:
- LinuxThreads
- Native POSIX Thread Library (NPTL)
- IBM’s Next Generation POSIX Threading Package (NGPT)
Describe a thread group in Linux.
In Linux, a thread group is basically a set of lightweight processes that implement a multithreaded application and act as a whole with regards to some system calls such as
getpid()
,kill()
, and_exit()
.
Process Descriptors and States (Q5-Q9)
What is meant by a process descriptor and explain its role?
The process descriptor is used to manage processes. It must contain all information related to a single process, such as:
- The process’s priority.
- Whether it is running on a CPU or blocked on an event.
- What address space has been assigned to it.
- Which files it is allowed to address.
With the help of a schematic, describe the Linux process descriptor structure
task_struct
.(A schematic diagram illustrating the fields and pointers within the
task_struct
is required here.)List the seven possible process states in the Linux OS and explain them.
The seven possible process states are:
TASK_RUNNING
TASK_INTERRUPTIBLE
TASK_UNINTERRUPTIBLE
TASK_STOPPED
TASK_TRACED
EXIT_ZOMBIE
EXIT_DEAD
Describe the following process states in Linux:
TASK_RUNNING
- The process is either executing on a CPU or waiting to be executed.
TASK_INTERRUPTIBLE
- The process is suspended (sleeping) until some condition becomes true. Raising a hardware interrupt, releasing a system resource the process is waiting for, or delivering a signal are examples of conditions that might wake up the process (put its state back to
TASK_RUNNING
). TASK_UNINTERRUPTIBLE
- Like
TASK_INTERRUPTIBLE
, except that delivering a signal to the sleeping process leaves its state unchanged. TASK_STOPPED
- Process execution has been stopped; the process enters this state after receiving a
SIGSTOP
,SIGTSTP
,SIGTTIN
, orSIGTTOU
signal. TASK_TRACED
- Process execution has been stopped by a debugger. When a process is being monitored by another (such as when a debugger executes a
ptrace()
system call to monitor a test program), each signal may put the process in theTASK_TRACED
state. EXIT_ZOMBIE
- Process execution is terminated, but the parent process has not yet issued a
wait4()
orwaitpid()
system call to return information about the dead process. Before thewait()
-like call is issued, the kernel cannot discard the data contained in the dead process descriptor because the parent might need it. There are otherwait()
-like library functions, such aswait3()
andwait()
, but in Linux, they are implemented by means of thewait4()
andwaitpid()
system calls. EXIT_DEAD
- The final state: the process is being removed by the system because the parent process has just issued a
wait4()
orwaitpid()
system call for it. Changing its state fromEXIT_ZOMBIE
toEXIT_DEAD
avoids race conditions due to other threads of execution that executewait()
-like calls on the same process.
State the purpose of the
TASK_UNINTERRUPTIBLE
state in a Linux process.It is like
TASK_INTERRUPTIBLE
, except that delivering a signal to the sleeping process leaves its state unchanged. This state is typically used when a process is waiting for I/O completion that cannot be interrupted.
Process Identification (PIDs) and Memory Structures (Q10-Q20)
State the two approaches used by the kernel to identify a process.
The two functions used are
set_task_state
andset_current_state
.What is meant by PID? Why is it required?
PID stands for Process ID. It is required to identify processes by means of a unique number.
What is the upper limit on the PID values in 32-bit architectures?
32,767.
What is the upper limit on the PID values in 64-bit architectures?
4,194,303.
State the purpose of a
pidmap_array
bitmap in Linux. Indicate the size of thepidmap_array
bitmap.The
pidmap_array
bitmap is used to denote which PIDs are currently assigned and which ones are free. Its size is 32,768 bits.Indicate which lightweight process becomes the thread group leader by default. What is the PID assigned to the thread group?
The identifier shared by the threads is the PID of the thread group leader, that is, the PID of the first lightweight process in the group; it is stored in the
tgid
field of the process descriptors.Since the kernel must be able to handle many processes at the same time, why are process descriptors stored in dynamic memory rather than in the memory area permanently assigned to the kernel?
Process descriptors are stored in dynamic memory because the kernel must be able to handle many processes simultaneously, and dynamic allocation allows flexible management of these structures as processes are created and terminated.
Describe the two different data structures assigned to every process by the kernel in a single per-process memory area. Indicate the length of this memory area and explain the data structures stored in this memory area with the help of a diagram showing both these data structures.
The two data structures are:
- A small data structure linked to the process descriptor, namely the
thread_info
structure. - The kernel mode process stack.
The length of this memory area is usually 8,192 bytes (two page frames). (A diagram would typically show the
thread_info
structure located at the bottom of the memory area, with the kernel stack growing upwards.)- A small data structure linked to the process descriptor, namely the
Explain the purpose of the following two different data structures assigned to every process by the kernel in a single per-process memory area. Indicate the size of each of these data structures.
a)
thread_info
structure:The
thread_info
structure is a small structure linked to the process descriptor. In the 80×86 architecture, the kernel can be configured at compilation time so that the memory area including the stack andthread_info
structure spans a single page frame (4,096 bytes).b) Kernel mode process stack:
A process in kernel mode accesses a stack contained in the kernel data segment, which is different from the stack used by the process in user mode. Because kernel control paths make little use of the stack, only a few thousand bytes of kernel stack are required. Therefore, 8 KB is ample space for the stack and the
thread_info
structure.State the key benefit in terms of efficiency offered by the kernel in providing a close association between the
thread_info
structure and the kernel mode stack.Because the
thread_info
structure is 52 bytes long, the kernel stack can expand up to 8,140 bytes. The kernel can easily obtain the address of thethread_info
structure of the process currently running on a CPU from the value of theesp
register. In fact, if thethread_union
structure is 8 KB (213 bytes) long, the kernel masks out the 13 least significant bits ofesp
to obtain the base address of thethread_info
structure. On the other hand, if thethread_union
structure is 4 KB long, the kernel masks out the 12 least significant bits ofesp
.State the purpose of the
current
macro in Linux.Most often, the kernel needs the address of the process descriptor rather than the address of the
thread_info
structure. To get the process descriptor pointer of the process currently running on a CPU, the kernel makes use of thecurrent
macro, which is essentially equivalent tocurrent_thread_info()->task
. Thecurrent
macro often appears in kernel code as a prefix to fields of the process descriptor. For example,current->pid
returns the process ID of the process currently running on the CPU.
Process Lists and Scheduling Structures (Q21-Q25)
Explain what is meant by the process list. Explain how the process list is implemented in Linux.
In Linux, the process list is implemented as a doubly linked list, a list that links together all existing process descriptors. The head of the process list is the
init_task
task_struct
descriptor; it is the process descriptor of the so-called Process 0 or the Swapper.- The
tasks->prev
field ofinit_task
points to thetasks
field of the process descriptor inserted last in the list. - The
set_links
andremove_links
macros are used to insert and to remove a process descriptor in the process list, respectively.
- The
With the help of a block diagram, show the special data structures used to implement the process list. Also indicate the processes the first two nodes in the data structure point to.
(A block diagram is required here, typically showing the circular doubly linked list structure anchored by
init_task
, pointing to Process 1 (Init) and the last inserted process.)Explain how the runqueue is implemented in Linux to achieve scheduler speedup to select the best runnable process in constant time.
Earlier Linux versions put all runnable processes in the same list called the runqueue. Modern Linux kernels (since O(1) and CFS schedulers) use more complex structures (like red-black trees or priority arrays) to ensure that the best runnable process can be selected in constant time, O(1).
With the help of a figure, illustrate the relationship between the parent and siblings of a group of processes.
For example, if Process P0 successively created P1, P2, and P3, and Process P3, in turn, created Process P4, the figure would illustrate P0 as the parent of P1, P2, and P3 (siblings), and P3 as the parent of P4.
Write short notes on the PID hash table and chained lists.
To speed up the search for processes, four hash tables have been introduced. This is because the process descriptor includes fields that represent different types of PID, and each type of PID requires its own hash table.
Linux uses chaining to handle colliding PIDs; each table entry is the head of a doubly linked list of colliding process descriptors.
Hashing with chaining is preferable to a linear transformation from PIDs to table indexes because, at any given instance, the number of processes in the system is usually far below 32,768 (the maximum number of allowed PIDs).
Wait Queues and Synchronization (Q29-Q34)
Explain the purpose of a wait queue in Linux.
The process state does not provide enough information to retrieve the process quickly, so it is necessary to introduce additional lists of processes called wait queues. A wait queue represents a set of sleeping processes, which are woken up by the kernel when some condition becomes true.
Describe the runqueue and wait queues in Linux.
The runqueue lists group all processes in a
TASK_RUNNING
state. When it comes to grouping processes in other states, the various states call for different types of treatment, with Linux opting for wait queues to manage processes waiting for specific events.Indicate the three important uses of wait queues in the kernel.
The three important uses are:
- Interrupt handling
- Process synchronization
- Timing
Describe how wait queues are implemented in Linux by providing the following two data structures –
struct __wait_queue_head
andstruct __wait_queue
– and clearly indicating the purpose of various fields in these data structures.1. Wait Queue Head (
wait_queue_head_t
):struct __wait_queue_head { spinlock_t lock; struct list_head task_list; }; typedef struct __wait_queue_head wait_queue_head_t;
- The
lock
spin lock protects the doubly linked lists from concurrent accesses, which could induce unpredictable results, as wait queues are modified by interrupt handlers as well as by major kernel functions. - The
task_list
field is the head of the list of waiting processes.
2. Wait Queue Element (
wait_queue_t
):struct __wait_queue { unsigned int flags; struct task_struct * task; wait_queue_func_t func; struct list_head task_list; }; typedef struct __wait_queue wait_queue_t;
- The
task
field stores the descriptor address of the sleeping process waiting for an event. - The
task_list
field contains the pointers that link this element to the list of processes waiting for the same event. - The
flags
field indicates the kind of sleeping process (exclusive or non-exclusive). - The
func
field specifies how the processes sleeping in the wait queue should be woken up.
- The
What is meant by the “thundering herd” problem? Explain how it is tackled.
The “thundering herd” problem occurs when multiple processes are woken up only to race for a resource that can be accessed by only one of them, with the result that the remaining processes must once more be put back to sleep.
This problem is tackled by classifying sleeping processes as exclusive or non-exclusive.
What are the two kinds of sleeping processes in a wait queue? Why is this classification required?
The two kinds of sleeping processes are:
- Exclusive processes: Denoted by the value 1 in the
flags
field of the corresponding wait queue element. These are selectively woken up by the kernel. - Non-exclusive processes: Denoted by the value 0 in the
flags
field. These are always woken up by the kernel when the event occurs.
This classification is required to mitigate the “thundering herd” problem by ensuring that only necessary processes are awakened.
- Exclusive processes: Denoted by the value 1 in the
Resource Limits and Context Switching (Q35-Q40)
Explain process resource limits in Linux and list the various resource limits specified.
Each process has an associated set of resource limits, which specify the amount of system resources it can use. These limits keep a user from overwhelming the system (its CPU, disk space, and so on). Resource limits include:
- CPU time (seconds)
- File size (blocks)
- Data segment size (bytes)
- Stack size (bytes)
- Core file size (blocks)
- Resident set size (bytes)
- Number of processes
- Number of open files
- Memory locked (bytes)
- Maximum number of file locks
State the purpose of the following commands in Unix/Linux:
ulimit -ha
- Displays all current hard resource limits.
ulimit -sa
- Displays all current soft resource limits.
getconf -a
- Displays all system configuration variables and their values.
getconf PAGE_SIZE
- Displays the size of a memory page in bytes for the current system architecture.
Describe process switch or context switch or task switch in Linux.
To control the execution of processes, the kernel must be able to suspend the execution of the process running on the CPU and resume the execution of some other process previously suspended. This activity goes variously by the names process switch, task switch, or context switch.
Explain hardware context in Linux.
The set of data that must be loaded into the registers before the process resumes its execution on the CPU is called the hardware context. In Linux, a part of the hardware context of a process is stored in the process descriptor, while the remaining part is saved in the kernel mode stack.
Write short notes on the following:
a) Task State Segment (TSS):
Linux uses hardware support offered by the 80×86 architecture and performs a process switch through a
jmp
instruction to the selector of the Task State Segment (TSS) descriptor of the next process. The 80×86 architecture includes a specific segment type called the TSS to store hardware contexts.b)
thread
field:Each process descriptor includes a field called
thread
of typethread_struct
, in which the kernel saves the hardware context whenever the process is being switched out.Describe how the kernel performs a process switch using the
schedule()
function.The
schedule()
function performs a process switch by executing two main steps:- Switching the Page Global Directory to install a new address space.
- Switching the kernel mode stack and the hardware context, which provides all the information needed by the kernel to execute the new process, including the CPU registers.
Process Creation and Termination (Q41-Q50)
Explain the three different mechanisms introduced by modern Unix kernels to solve the problem of duplicating address space. Also indicate the system call associated with each one of these mechanisms to create a process.
Copy-on-Write (CoW) Technique (associated with
fork()
): Allows both the parent and the child to read the same physical pages. Whenever either one tries to write on a physical page, the kernel copies its contents into a new physical page that is assigned to the writing process.Lightweight Processes (associated with
clone()
): Allow both the parent and the child to share many per-process kernel data structures, such as the paging tables (and therefore the entire user mode address space), the open file tables, and the signal dispositions.vfork()
System Call: Creates a process that shares the memory address space of its parent. To prevent the parent from overwriting data needed by the child, the parent’s execution is blocked until the child exits or executes a new program.
State the three system calls used to create a process. Why are these system calls required?
The three system calls are
clone()
,fork()
, andvfork()
. They are required to provide different levels of resource sharing and execution control between the parent and child processes, optimizing for various scenarios like multithreading (clone()
) or immediate execution of a new program (vfork()
).Write the system call used to create a lightweight process in Linux.
Lightweight processes are created in Linux by using a function named
clone()
, which uses the following parameters:fn
: Specifies a function to be executed by the new process; when the function returns, the child terminates. The function returns an integer, which represents the exit code for the child process.arg
: Points to data passed to thefn()
function.flags
: Miscellaneous information. The low byte specifies the signal number to be sent to the parent process when the child terminates; theSIGCHLD
signal is generally selected.child_stack
: Specifies the user mode stack pointer to be assigned to theesp
register of the child process. The invoking process (the parent) should always allocate a new stack for the child.tls
: Specifies the address of a data structure that defines a thread local storage segment for the new lightweight process. Meaningful only if theCLONE_SETTLS
flag is set.ptid
: Specifies the address of a user mode variable of the parent process that will hold the PID of the new lightweight process. Meaningful only if theCLONE_PARENT_SETTID
flag is set.ctid
: Specifies the address of a user mode variable of the new lightweight process that will hold the PID of such process. Meaningful only if theCLONE_CHILD_SETTID
flag is set.
Write short notes on:
a) Process 0:
The ancestor of all processes, called Process 0, the idle process, or, for historical reasons, the Swapper process, is a kernel thread created from scratch during the initialization phase of Linux.
b) Process 1:
The kernel thread created by Process 0 executes the
init()
function, which in turn completes the initialization of the kernel. Theninit()
invokes theexecve()
system call to load the executable programinit
(orsystemd
/equivalent).Describe the operations performed by the following functions in Linux:
a)
do_fork()
:The
do_fork()
function makes use of an auxiliary function calledcopy_process()
to set up the process descriptor and any other kernel data structure required for the child’s execution.b)
copy_process()
:The
copy_process()
function is the core routine used bydo_fork()
(and thusfork()
,vfork()
, andclone()
) to duplicate the parent’s process descriptor and resources for the new child process. The Swapper process running on CPU 0 initializes the kernel data structures, then enables the other CPUs and creates the additional swapper processes by means of thecopy_process()
function, passing to it the value 0 as the new PID.What are kernel threads? Explain how kernel threads are created in Linux.
Kernel threads run only in kernel mode, while regular processes run alternately in kernel mode and in user mode. Kernel threads are created using the
kthread_create()
orkernel_thread()
functions. Thekernel_thread()
function receives as parameters the address of the kernel function to be executed (fn
), the argument to be passed to that function (arg
), and a set ofclone
flags (flags
). This function essentially invokesdo_fork()
.State the purpose of the following functions in Linux:
a)
kernel_thread()
:The function creates a new kernel thread.
b)
copy_process()
:The function duplicates the process descriptor and resources for a new process or thread.
c)
exit()
:The system call terminates a single process, regardless of any other process in the thread group of the victim.
d)
exit_group()
:The system call terminates a full thread group, that is, a whole multithreaded application.
Describe how processes are created and terminated/destroyed in Linux.
Processes are created by using a function named
clone()
(which underliesfork()
andvfork()
) and terminated/destroyed by_exit()
orexit_group()
.What is an orphan process? How is an orphan process handled in Linux to eliminate memory leaks?
An orphan process is a process whose parent has terminated before it has. To eliminate memory leaks, orphan processes are reparented to become children of the
init
process (Process 1).What happens if a parent process terminates before its children? How is this problem solved to eliminate memory leaks?
If a parent process terminates before its children, the system could be flooded with zombie processes whose process descriptors would stay forever in RAM. This problem is solved by forcing all orphan processes to become children of the
init
process. In this way, theinit
process will destroy the zombies while checking for the termination of one of its legitimate children through await()
-like system call.