Linux Process Management: Data Structures and System Calls

Linux Process Management: Basic Data Structures and System Calls (Chapter 3)

Fundamental Process Concepts (Q1-Q4)

  1. What is a process?

    A process is a program in execution.

  2. What are lightweight processes in Linux?

    Lightweight processes are processes which offer better support for multithreaded applications.

  3. Explain what is meant by a multithreaded application. Describe how multithreaded applications are implemented in Linux. Give three examples of POSIX-compliant pthread libraries.

    A straightforward way to implement multithreaded applications is to associate a lightweight process with each thread.

    In this way, the threads can access the same set of application data structures by simply sharing the same memory address space, the same set of open files, and so on. At the same time, each thread can be scheduled independently by the kernel so that one may sleep while another remains runnable.

    Examples of POSIX-compliant pthread libraries that use Linux’s lightweight processes are:

    • LinuxThreads
    • Native POSIX Thread Library (NPTL)
    • IBM’s Next Generation POSIX Threading Package (NGPT)
  4. Describe a thread group in Linux.

    In Linux, a thread group is basically a set of lightweight processes that implement a multithreaded application and act as a whole with regards to some system calls such as getpid(), kill(), and _exit().

Process Descriptors and States (Q5-Q9)

  1. What is meant by a process descriptor and explain its role?

    The process descriptor is used to manage processes. It must contain all information related to a single process, such as:

    • The process’s priority.
    • Whether it is running on a CPU or blocked on an event.
    • What address space has been assigned to it.
    • Which files it is allowed to address.
  2. With the help of a schematic, describe the Linux process descriptor structure task_struct.

    (A schematic diagram illustrating the fields and pointers within the task_struct is required here.)

  3. List the seven possible process states in the Linux OS and explain them.

    The seven possible process states are:

    • TASK_RUNNING
    • TASK_INTERRUPTIBLE
    • TASK_UNINTERRUPTIBLE
    • TASK_STOPPED
    • TASK_TRACED
    • EXIT_ZOMBIE
    • EXIT_DEAD
  4. Describe the following process states in Linux:

    TASK_RUNNING
    The process is either executing on a CPU or waiting to be executed.
    TASK_INTERRUPTIBLE
    The process is suspended (sleeping) until some condition becomes true. Raising a hardware interrupt, releasing a system resource the process is waiting for, or delivering a signal are examples of conditions that might wake up the process (put its state back to TASK_RUNNING).
    TASK_UNINTERRUPTIBLE
    Like TASK_INTERRUPTIBLE, except that delivering a signal to the sleeping process leaves its state unchanged.
    TASK_STOPPED
    Process execution has been stopped; the process enters this state after receiving a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal.
    TASK_TRACED
    Process execution has been stopped by a debugger. When a process is being monitored by another (such as when a debugger executes a ptrace() system call to monitor a test program), each signal may put the process in the TASK_TRACED state.
    EXIT_ZOMBIE
    Process execution is terminated, but the parent process has not yet issued a wait4() or waitpid() system call to return information about the dead process. Before the wait()-like call is issued, the kernel cannot discard the data contained in the dead process descriptor because the parent might need it. There are other wait()-like library functions, such as wait3() and wait(), but in Linux, they are implemented by means of the wait4() and waitpid() system calls.
    EXIT_DEAD
    The final state: the process is being removed by the system because the parent process has just issued a wait4() or waitpid() system call for it. Changing its state from EXIT_ZOMBIE to EXIT_DEAD avoids race conditions due to other threads of execution that execute wait()-like calls on the same process.
  5. State the purpose of the TASK_UNINTERRUPTIBLE state in a Linux process.

    It is like TASK_INTERRUPTIBLE, except that delivering a signal to the sleeping process leaves its state unchanged. This state is typically used when a process is waiting for I/O completion that cannot be interrupted.

Process Identification (PIDs) and Memory Structures (Q10-Q20)

  1. State the two approaches used by the kernel to identify a process.

    The two functions used are set_task_state and set_current_state.

  2. What is meant by PID? Why is it required?

    PID stands for Process ID. It is required to identify processes by means of a unique number.

  3. What is the upper limit on the PID values in 32-bit architectures?

    32,767.

  4. What is the upper limit on the PID values in 64-bit architectures?

    4,194,303.

  5. State the purpose of a pidmap_array bitmap in Linux. Indicate the size of the pidmap_array bitmap.

    The pidmap_array bitmap is used to denote which PIDs are currently assigned and which ones are free. Its size is 32,768 bits.

  6. Indicate which lightweight process becomes the thread group leader by default. What is the PID assigned to the thread group?

    The identifier shared by the threads is the PID of the thread group leader, that is, the PID of the first lightweight process in the group; it is stored in the tgid field of the process descriptors.

  7. Since the kernel must be able to handle many processes at the same time, why are process descriptors stored in dynamic memory rather than in the memory area permanently assigned to the kernel?

    Process descriptors are stored in dynamic memory because the kernel must be able to handle many processes simultaneously, and dynamic allocation allows flexible management of these structures as processes are created and terminated.

  8. Describe the two different data structures assigned to every process by the kernel in a single per-process memory area. Indicate the length of this memory area and explain the data structures stored in this memory area with the help of a diagram showing both these data structures.

    The two data structures are:

    1. A small data structure linked to the process descriptor, namely the thread_info structure.
    2. The kernel mode process stack.

    The length of this memory area is usually 8,192 bytes (two page frames). (A diagram would typically show the thread_info structure located at the bottom of the memory area, with the kernel stack growing upwards.)

  9. Explain the purpose of the following two different data structures assigned to every process by the kernel in a single per-process memory area. Indicate the size of each of these data structures.

    a) thread_info structure:

    The thread_info structure is a small structure linked to the process descriptor. In the 80×86 architecture, the kernel can be configured at compilation time so that the memory area including the stack and thread_info structure spans a single page frame (4,096 bytes).

    b) Kernel mode process stack:

    A process in kernel mode accesses a stack contained in the kernel data segment, which is different from the stack used by the process in user mode. Because kernel control paths make little use of the stack, only a few thousand bytes of kernel stack are required. Therefore, 8 KB is ample space for the stack and the thread_info structure.

  10. State the key benefit in terms of efficiency offered by the kernel in providing a close association between the thread_info structure and the kernel mode stack.

    Because the thread_info structure is 52 bytes long, the kernel stack can expand up to 8,140 bytes. The kernel can easily obtain the address of the thread_info structure of the process currently running on a CPU from the value of the esp register. In fact, if the thread_union structure is 8 KB (213 bytes) long, the kernel masks out the 13 least significant bits of esp to obtain the base address of the thread_info structure. On the other hand, if the thread_union structure is 4 KB long, the kernel masks out the 12 least significant bits of esp.

  11. State the purpose of the current macro in Linux.

    Most often, the kernel needs the address of the process descriptor rather than the address of the thread_info structure. To get the process descriptor pointer of the process currently running on a CPU, the kernel makes use of the current macro, which is essentially equivalent to current_thread_info()->task. The current macro often appears in kernel code as a prefix to fields of the process descriptor. For example, current->pid returns the process ID of the process currently running on the CPU.

Process Lists and Scheduling Structures (Q21-Q25)

  1. Explain what is meant by the process list. Explain how the process list is implemented in Linux.

    In Linux, the process list is implemented as a doubly linked list, a list that links together all existing process descriptors. The head of the process list is the init_task task_struct descriptor; it is the process descriptor of the so-called Process 0 or the Swapper.

    • The tasks->prev field of init_task points to the tasks field of the process descriptor inserted last in the list.
    • The set_links and remove_links macros are used to insert and to remove a process descriptor in the process list, respectively.
  2. With the help of a block diagram, show the special data structures used to implement the process list. Also indicate the processes the first two nodes in the data structure point to.

    (A block diagram is required here, typically showing the circular doubly linked list structure anchored by init_task, pointing to Process 1 (Init) and the last inserted process.)

  3. Explain how the runqueue is implemented in Linux to achieve scheduler speedup to select the best runnable process in constant time.

    Earlier Linux versions put all runnable processes in the same list called the runqueue. Modern Linux kernels (since O(1) and CFS schedulers) use more complex structures (like red-black trees or priority arrays) to ensure that the best runnable process can be selected in constant time, O(1).

  4. With the help of a figure, illustrate the relationship between the parent and siblings of a group of processes.

    For example, if Process P0 successively created P1, P2, and P3, and Process P3, in turn, created Process P4, the figure would illustrate P0 as the parent of P1, P2, and P3 (siblings), and P3 as the parent of P4.

  5. Write short notes on the PID hash table and chained lists.

    To speed up the search for processes, four hash tables have been introduced. This is because the process descriptor includes fields that represent different types of PID, and each type of PID requires its own hash table.

    Linux uses chaining to handle colliding PIDs; each table entry is the head of a doubly linked list of colliding process descriptors.

    Hashing with chaining is preferable to a linear transformation from PIDs to table indexes because, at any given instance, the number of processes in the system is usually far below 32,768 (the maximum number of allowed PIDs).

Wait Queues and Synchronization (Q29-Q34)

  1. Explain the purpose of a wait queue in Linux.

    The process state does not provide enough information to retrieve the process quickly, so it is necessary to introduce additional lists of processes called wait queues. A wait queue represents a set of sleeping processes, which are woken up by the kernel when some condition becomes true.

  2. Describe the runqueue and wait queues in Linux.

    The runqueue lists group all processes in a TASK_RUNNING state. When it comes to grouping processes in other states, the various states call for different types of treatment, with Linux opting for wait queues to manage processes waiting for specific events.

  3. Indicate the three important uses of wait queues in the kernel.

    The three important uses are:

    1. Interrupt handling
    2. Process synchronization
    3. Timing
  4. Describe how wait queues are implemented in Linux by providing the following two data structures – struct __wait_queue_head and struct __wait_queue – and clearly indicating the purpose of various fields in these data structures.

    1. Wait Queue Head (wait_queue_head_t):

    struct __wait_queue_head {
      spinlock_t lock;
      struct list_head task_list;
    };
    typedef struct __wait_queue_head wait_queue_head_t;
    • The lock spin lock protects the doubly linked lists from concurrent accesses, which could induce unpredictable results, as wait queues are modified by interrupt handlers as well as by major kernel functions.
    • The task_list field is the head of the list of waiting processes.

    2. Wait Queue Element (wait_queue_t):

    struct __wait_queue {
      unsigned int flags;
      struct task_struct * task;
      wait_queue_func_t func;
      struct list_head task_list;
    };
    typedef struct __wait_queue wait_queue_t;
    • The task field stores the descriptor address of the sleeping process waiting for an event.
    • The task_list field contains the pointers that link this element to the list of processes waiting for the same event.
    • The flags field indicates the kind of sleeping process (exclusive or non-exclusive).
    • The func field specifies how the processes sleeping in the wait queue should be woken up.
  5. What is meant by the “thundering herd” problem? Explain how it is tackled.

    The “thundering herd” problem occurs when multiple processes are woken up only to race for a resource that can be accessed by only one of them, with the result that the remaining processes must once more be put back to sleep.

    This problem is tackled by classifying sleeping processes as exclusive or non-exclusive.

  6. What are the two kinds of sleeping processes in a wait queue? Why is this classification required?

    The two kinds of sleeping processes are:

    1. Exclusive processes: Denoted by the value 1 in the flags field of the corresponding wait queue element. These are selectively woken up by the kernel.
    2. Non-exclusive processes: Denoted by the value 0 in the flags field. These are always woken up by the kernel when the event occurs.

    This classification is required to mitigate the “thundering herd” problem by ensuring that only necessary processes are awakened.

Resource Limits and Context Switching (Q35-Q40)

  1. Explain process resource limits in Linux and list the various resource limits specified.

    Each process has an associated set of resource limits, which specify the amount of system resources it can use. These limits keep a user from overwhelming the system (its CPU, disk space, and so on). Resource limits include:

    • CPU time (seconds)
    • File size (blocks)
    • Data segment size (bytes)
    • Stack size (bytes)
    • Core file size (blocks)
    • Resident set size (bytes)
    • Number of processes
    • Number of open files
    • Memory locked (bytes)
    • Maximum number of file locks
  2. State the purpose of the following commands in Unix/Linux:

    ulimit -ha
    Displays all current hard resource limits.
    ulimit -sa
    Displays all current soft resource limits.
    getconf -a
    Displays all system configuration variables and their values.
    getconf PAGE_SIZE
    Displays the size of a memory page in bytes for the current system architecture.
  3. Describe process switch or context switch or task switch in Linux.

    To control the execution of processes, the kernel must be able to suspend the execution of the process running on the CPU and resume the execution of some other process previously suspended. This activity goes variously by the names process switch, task switch, or context switch.

  4. Explain hardware context in Linux.

    The set of data that must be loaded into the registers before the process resumes its execution on the CPU is called the hardware context. In Linux, a part of the hardware context of a process is stored in the process descriptor, while the remaining part is saved in the kernel mode stack.

  5. Write short notes on the following:

    a) Task State Segment (TSS):

    Linux uses hardware support offered by the 80×86 architecture and performs a process switch through a jmp instruction to the selector of the Task State Segment (TSS) descriptor of the next process. The 80×86 architecture includes a specific segment type called the TSS to store hardware contexts.

    b) thread field:

    Each process descriptor includes a field called thread of type thread_struct, in which the kernel saves the hardware context whenever the process is being switched out.

  6. Describe how the kernel performs a process switch using the schedule() function.

    The schedule() function performs a process switch by executing two main steps:

    1. Switching the Page Global Directory to install a new address space.
    2. Switching the kernel mode stack and the hardware context, which provides all the information needed by the kernel to execute the new process, including the CPU registers.

Process Creation and Termination (Q41-Q50)

  1. Explain the three different mechanisms introduced by modern Unix kernels to solve the problem of duplicating address space. Also indicate the system call associated with each one of these mechanisms to create a process.

    1. Copy-on-Write (CoW) Technique (associated with fork()): Allows both the parent and the child to read the same physical pages. Whenever either one tries to write on a physical page, the kernel copies its contents into a new physical page that is assigned to the writing process.

    2. Lightweight Processes (associated with clone()): Allow both the parent and the child to share many per-process kernel data structures, such as the paging tables (and therefore the entire user mode address space), the open file tables, and the signal dispositions.

    3. vfork() System Call: Creates a process that shares the memory address space of its parent. To prevent the parent from overwriting data needed by the child, the parent’s execution is blocked until the child exits or executes a new program.

  2. State the three system calls used to create a process. Why are these system calls required?

    The three system calls are clone(), fork(), and vfork(). They are required to provide different levels of resource sharing and execution control between the parent and child processes, optimizing for various scenarios like multithreading (clone()) or immediate execution of a new program (vfork()).

  3. Write the system call used to create a lightweight process in Linux.

    Lightweight processes are created in Linux by using a function named clone(), which uses the following parameters:

    • fn: Specifies a function to be executed by the new process; when the function returns, the child terminates. The function returns an integer, which represents the exit code for the child process.
    • arg: Points to data passed to the fn() function.
    • flags: Miscellaneous information. The low byte specifies the signal number to be sent to the parent process when the child terminates; the SIGCHLD signal is generally selected.
    • child_stack: Specifies the user mode stack pointer to be assigned to the esp register of the child process. The invoking process (the parent) should always allocate a new stack for the child.
    • tls: Specifies the address of a data structure that defines a thread local storage segment for the new lightweight process. Meaningful only if the CLONE_SETTLS flag is set.
    • ptid: Specifies the address of a user mode variable of the parent process that will hold the PID of the new lightweight process. Meaningful only if the CLONE_PARENT_SETTID flag is set.
    • ctid: Specifies the address of a user mode variable of the new lightweight process that will hold the PID of such process. Meaningful only if the CLONE_CHILD_SETTID flag is set.
  4. Write short notes on:

    a) Process 0:

    The ancestor of all processes, called Process 0, the idle process, or, for historical reasons, the Swapper process, is a kernel thread created from scratch during the initialization phase of Linux.

    b) Process 1:

    The kernel thread created by Process 0 executes the init() function, which in turn completes the initialization of the kernel. Then init() invokes the execve() system call to load the executable program init (or systemd/equivalent).

  5. Describe the operations performed by the following functions in Linux:

    a) do_fork():

    The do_fork() function makes use of an auxiliary function called copy_process() to set up the process descriptor and any other kernel data structure required for the child’s execution.

    b) copy_process():

    The copy_process() function is the core routine used by do_fork() (and thus fork(), vfork(), and clone()) to duplicate the parent’s process descriptor and resources for the new child process. The Swapper process running on CPU 0 initializes the kernel data structures, then enables the other CPUs and creates the additional swapper processes by means of the copy_process() function, passing to it the value 0 as the new PID.

  6. What are kernel threads? Explain how kernel threads are created in Linux.

    Kernel threads run only in kernel mode, while regular processes run alternately in kernel mode and in user mode. Kernel threads are created using the kthread_create() or kernel_thread() functions. The kernel_thread() function receives as parameters the address of the kernel function to be executed (fn), the argument to be passed to that function (arg), and a set of clone flags (flags). This function essentially invokes do_fork().

  7. State the purpose of the following functions in Linux:

    a) kernel_thread():

    The function creates a new kernel thread.

    b) copy_process():

    The function duplicates the process descriptor and resources for a new process or thread.

    c) exit():

    The system call terminates a single process, regardless of any other process in the thread group of the victim.

    d) exit_group():

    The system call terminates a full thread group, that is, a whole multithreaded application.

  8. Describe how processes are created and terminated/destroyed in Linux.

    Processes are created by using a function named clone() (which underlies fork() and vfork()) and terminated/destroyed by _exit() or exit_group().

  9. What is an orphan process? How is an orphan process handled in Linux to eliminate memory leaks?

    An orphan process is a process whose parent has terminated before it has. To eliminate memory leaks, orphan processes are reparented to become children of the init process (Process 1).

  10. What happens if a parent process terminates before its children? How is this problem solved to eliminate memory leaks?

    If a parent process terminates before its children, the system could be flooded with zombie processes whose process descriptors would stay forever in RAM. This problem is solved by forcing all orphan processes to become children of the init process. In this way, the init process will destroy the zombies while checking for the termination of one of its legitimate children through a wait()-like system call.