Linux Operating System Fundamentals

Posted on Jun 19, 2024 in Computers

Operating System (OS)

Programs that interface between hardware and users.

Unix Philosophy

  • Everything is a “file”.
  • No news is good news.
  • Programs are designed to work together.

Unix Problems

  • Too many ways to do things.
  • Overlapping functions and utilities.
  • Different syntaxes across UNIX flavors.

Examples

  • echo a{d,c,b}e = ade\nace\nabe
  • ~ = /home/username
  • $var = $HOME
  • $(command) = `command`
  • $((expression)) = a
  • * = 0 or 1 char
  • ? = 1 char
  • ' ' = literal
  • " " = literal except $ & ‘
  • Stdin = command
  • Stdout = command >(! To ignore noclobber)(>) outfile
  • Stderr = command >(! To ignore noclobber) outfile
  • command &>(>) outfile = stdout/stderr to file
  • (command > file1) >& file2 = send stdout to file1 and stderr to file2
  • process1 | process2 | process3 = process1 > file1 & process2 file2 & process3
  • chmod = Owner/Group/Public read=1 write=2 execute=4 (for default use umask)

Process States

  • Active: Process currently executed.
  • Waiting: Waiting for a resource.
  • Ready: Waiting for CPU attention.
  • Halted: Finished/stopped.
  • Terminated: OS removing resources.

Symlinks

ln -s original new (perm from orig file, different dir, logical link, works on dir, naruto shadow clone)

Hard Links

ln, connection between file data and dir entry (points to file, works on local fs, naruto duplicate, deleting = removing last link)

Linux Virtual File System (VFS)

  • Inode: File information.
  • File
  • FS Object: Filesystem as a whole.

Ext2 Filesystem

Superblock

All global filesystem information is kept here (total number of blocks in fs, last error check time). If damaged, the filesystem is destroyed. A duplicate block is placed in each group of the filesystem.

Group Structures

Stores the location of the inode table, bitmaps, and the end of the data block area.

Inode

Each file = 1 inode (metadata: access and modification time, permissions, owner, type, where it is stored), points to the location on disk via 15 pointers:

  • First 12 = direct pointers
  • 13th = indirect pointer
  • 14th = doubly indirect pointer
  • 15th = triple indirect pointer

Extents

A set of blocks logically contiguous within a file.

Vi Editor

:range s/FindMe/ReplaceME/gc where:

  • range = % (all lines), . (current line | default), $ (the last line), {start-n},{end-n} (eg. 100,200s)
  • g (optional) = not present: substitute only the first match
  • c (optional) = ask for confirmation
  • i (optional) = case-insensitive

Terminal I/O

Canonical Mode (Cooked)

  • Processes terminal input line-by-line.
  • Input is returned one line at a time, with various line-ending characters like NL, EOL, EOF, ICRNL flag (maps CR to NL).
  • Reading can be affected by signals unless SA_RESTART is used.

Non-Canonical Mode (Raw)

  • Input characters are not assembled into lines.
  • Time limits can be set for read operations.
  • Special characters such as ERASE, KILL, and EOF can be disabled.

STTY

Command used to set the terminal line discipline parameters.

Flagsets

  • Input Flags (c_iflag): Control the input of characters.
  • Output Flags (c_oflag): Control the output by the terminal driver.
  • Control Flags (c_cflag): Control the behavior of asynchronous serial transmission lines.
  • Local Flags (c_lflag): Affect the interface between the driver and the user, like echoing and erasing characters.
  • The c_cc Array: Defines special control characters like EOF, EOL, ERASE, INTR, etc.

Terminal Driver Programming

  • Functions like tcgetattr, tcsetattr are used to get and set terminal attributes.
  • Options like TCSANOW, TCSADRAIN, and TCSAFLUSH specify when changes take effect.

struct termios term; tcgetattr(0, &term); term.c_lflag &= ~(ICANON); term.c_cc[VMIN] = 3; term.c_cc[VTIME] = 0; tcsetattr(0, TCSANOW, &term);

Time Limits

  • MIN > 0 and TIME > 0: Read returns after MIN bytes are read or TIME (in tenths of a second) elapses after the first byte. If no input is provided, read blocks indefinitely.
  • MIN > 0 and TIME == 0: No time limit is imposed on the read. It reads until at least MIN bytes are received, potentially blocking forever if MIN bytes are never received.
  • MIN == 0 and TIME > 0: The timer starts as soon as the read is called. Read returns after TIME tenths of a second elapse or a single byte is received.
  • MIN == 0 and TIME == 0: If data is available, the number of bytes requested (or all available data if less) are returned. If no data is available, read returns 0 immediately.

Options

  • TCSANOW: Immediately.
  • TCSADRAIN: After all output has been transmitted.
  • TCSAFLUSH: Unread input data is also discarded.

Window Size

  • Structure winsize holds terminal window dimensions.
  • ioctl calls TIOCGWINSZ and TIOCSWINSZ get and set window size, with SIGWINCH signal sent on changes.

Signals

  • Signals like SIGFPE, SIGSEV for errors, and SIGKILL, SIGINT for interruptions.
  • Handling signals using signal() to specify default actions, ignore signals, or execute custom handlers.

Filesizes

Extents represent a contiguous block range allocated to a file.
Instead of storing individual block information in inodes, extents group contiguous blocks together, reducing the metadata footprint.
This allows extents-based file systems to handle much larger file systems than the traditional inode-based approach of ext3.

Advantages

  • Faster read/write due to reduced fragmentation.
  • Larger file system (PB).
  • Reduced metadata overhead; improved performance; especially for large files.

Root Account Usage

  • Only use root for tasks requiring elevated privileges.
  • Avoid logging in as root except from the terminal.
  • Use ‘su’ or ‘sudo’ for performing root actions.

System Changes

  • Ensure all actions are reversible; take backups and create restore points.
  • Inform management before irreversible changes.
  • Test and verify the source of third-party software before running as root.

Documentation and Coordination

  • Document all actions for troubleshooting and auditing.
  • Coordinate changes with other administrators to avoid conflicts.

Filesystem Management

  • Use sync to flush buffers to disk.
  • Use fsck to check and repair filesystems.
  • Mount filesystems properly and manage disks by partitioning.

Kernel and Modules

  • Linux supports loadable kernel modules in /lib/modules.
  • Manage modules with modprobe, lsmod, and modules.conf.

Processes

  • Understand process states: active, waiting, ready, halted, terminated.
  • Manage processes with kill, knowing signals like SIGTERM and SIGKILL.
  • Use cron for scheduled tasks with crontab files in /var/spool/cron.

User Management

  • Commands: useradd, usermod, userdel, groupadd, groupmod, groupdel.
  • Passwords: managed with passwd, defaults in /etc/login.defs and /etc/default/useradd.
  • /etc/skel directory templates new user home directories.

Permissions

  • Use setuid and sticky bits for security on executables and directories.
  • chmod: Owner/Group/Public read=1, write=2, execute=4.

Runlevels

  • Understand runlevels 0 (halt), 1 (single-user), 3 (multi-user with networking), 5 (multi-user with X), and 6 (reboot).
  • /sbin/shutdown for graceful system shutdowns.
  • init process (PID 1) starts all processes, handles runlevels via /etc/inittab.

Important Directories and Files

  • /boot directory contains the kernel, typically named “vmlinuz”.
  • The init process is critical, using /etc/inittab file.

Commands

  • lsmod (list modules), insmod/rmmod (insert/remove modules).
  • tail -f (monitor file changes), mount (mount filesystem), du (disk usage).
  • fdisk (partition management), df (disk space), top (process table).
  • vmstat (virtual memory stats), free (memory usage), uname (kernel info).
  • chmod (change file permissions), dmesg (kernel log), stty (configure terminal settings).

Function Pointers

Defining function pointer:

typedef int (*MyType)(float, int); MyType ptr = &myFunction;

Interacting with Hardware Devices

DMA (Direct Memory Access): High-throughput I/O, RAM region access by CPU and device. To use: setup DMA channel, tell DMA controller to start, register interrupt service routine, controller interrupt after transfer complete.

Device Driver Types

  • Character Device: Stream of bytes, read and write directly without buffering, e.g., monitor, keyboard.
  • Block Device: Read and write in multiple blocks, e.g., disk, USB, DVD.
  • Network Device: Handled in the networking stack.

Device File

Interface for character/block devices (one file = one device).

/dev

Special directory in Linux for all device files.

Old Way

Major (device type)/minor (individual device)

New Way

Device file system

Common Block Device Layer

Common functions for all block devices in Linux (uniform interface, efficient buffer management, read-ahead, block I/O operations scheduling), generates and queues actual I/O operations in a request queue.

Regex Code

wFpLk+0wPdZkQAAAABJRU5ErkJggg==

Fork Example

pid_t parent = getpid();
pid_t pid = fork();
if (pid == -1)
{
// error, failed to fork()
}
else if (pid > 0)
{
int status;
waitpid(pid, &status, 0);
}
else
{
// we are the child
execve(...);
_exit(EXIT_FAILURE);   // exec never returns
}