Core Concepts in Operating Systems, Memory, and Cloud Computing

Posted on Oct 7, 2025 in Computers

System Architecture Fundamentals

Load-Store Architecture

The most common approach in processing is the load-store architecture.

Registers

Registers are tiny local memory (often called “scratch space”) on the processor into which instructions and data are copied.

Memory Management Concepts

Key concepts related to efficient memory utilization:

  • Caching: Buffering a copy of bytes (instructions and/or data) from a lower level at a higher level to exploit locality.
  • Prefetching: Preemptively retrieving bytes (typically data) from addresses not explicitly requested yet by the program.
  • Spill/Miss/Fault: Data needed for the program is not yet available at a higher level; it needs to be retrieved from a lower level.
    • Register Spill: Data moves from register to cache.
    • Cache Miss: Data moves from cache to main memory.
    • “Page” Fault: Data moves from main memory to disk.
  • Hit: Data needed is already available at the higher level.
  • Cache Replacement Policy: When new data needs to be loaded to a higher level, this policy determines which old data to evict to make room. Many policies exist with different properties.

Memory Hierarchy in Dask Context

  • Dask DataFrame automatically manages Disk versus DRAM for you.
  • Full data sits on Disk, brought to DRAM upon compute().
  • Dask stages out computations using Pandas.

Operating Systems (OS)

An OS is a large set of interrelated programs that make it easier for applications and user-written programs to use computer hardware effectively, efficiently, and securely. Without an OS, computer users must speak machine code!

The Abstraction of a Process

High-level steps the OS takes to initiate a process:

  1. Create a process (get Process ID; add to Process List).
  2. Assign part of DRAM to the process, known as its Address Space.
  3. Load code and static data (if applicable) to that space.
  4. Set up the inputs needed to run the program’s main() function.
  5. Update the process’s State to Ready.
  6. When the process is scheduled (Running), the OS temporarily hands off control to the process to run the show.
  7. Eventually, the process finishes or runs Destroy.

Virtualization

Each hardware resource is treated as a virtual entity that the OS can divide up and share among processes in a controlled way.

Limited Direct Execution

  • This is the OS mechanism used to time-share the CPU and preempt a process to run a different one, also known as a context switch.
  • A Scheduling policy tells the OS what time-sharing strategy to use.
  • Processes must also transfer control to the OS for “privileged” operations (e.g., I/O) via the System Calls API.

Concurrency and Multiprocessing

Multiprocessing: Different processes run on different cores (or entire CPUs) simultaneously.

Threads

A thread is a generalization of the OS’s Process abstraction.

  • A program spawns many threads; each runs parts of the program’s computations simultaneously.
  • Multithreading: The same core is used by many threads.

Issues in Multithreaded Programs with Shared Data

  • Cache coherence
  • Locking; deadlocks
  • Complex scheduling

Partitioning or replication of data simplifies concurrency.

Data Storage and Management

Filesystem

The part of the OS that helps programs create, manage, and delete files on disk (secondary storage).

  • File Descriptor (fd): An OS-assigned positive integer identifier/reference for a file’s virtual object that a process can use.
  • File Handle: A Programming Language’s (PL) abstraction built on top of a file descriptor.

Database Concepts

  • Database: An organized collection of interrelated data.
  • Data Model: An abstract model defining the organization of data in a formal (mathematically precise) way.
    • Logical level: Data model for higher-level reasoning.
    • Physical level: How bytes are layered on top of files.

Virtualization of DRAM with Pages

  • Page: An abstraction of fixed-size chunks of memory/storage. This makes it easier to virtualize and manage DRAM.
  • Page Frame: A virtual slot in DRAM used to hold a page’s content.
  • A Process’s Address Space: A slice of virtualized DRAM assigned exclusively to that process.

Persistent Memory Technologies

  • Persistent Memory (PMEM): A marketing term for large DRAM that is backed up by battery power.
  • Non-Volatile RAM (NVRAM): A popular term for a DRAM-like device that is genuinely non-volatile (requires no battery).

Data Organization on Disk

  • Disk space is organized into files.
  • Files are made up of disk pages, also known as blocks.

Magnetic Disk Quirks

Key Principle: The Sequential versus Random Access Dichotomy.

Cloud Computing Infrastructure

Core Cloud Principles

  • Manageability: Managing hardware is not the user’s problem.
  • Pay-as-you-go: Fine-grained pricing economics based on actual usage (granularity: seconds to years).
  • Elasticity: The ability to dynamically add or reduce capacity based on the actual workload’s demand.

Infrastructure as a Service (IaaS)

Compute Services

  • Elastic Compute Cloud (EC2)
  • Elastic Container Service (ECS)
  • Serverless compute engines: Fargate (serverless containers), Lambda (serverless functions)

Storage Services

  • Simple Storage Service (S3)
  • Elastic Block Store (EBS)
  • Elastic File System (EFS)
  • Glacier (storage classes)

Networking Services

  • CloudFront (low latency content delivery)
  • Virtual Private Cloud (VPC)

Platform as a Service (PaaS)

  • Database/Analytics Systems: Aurora, Redshift, Neptune, ElastiCache, DynamoDB, Timestream, EMR, Athena
  • Blockchain: QLDB
  • IoT: Greengrass
  • ML/AI: SageMaker*

Software as a Service (SaaS)

  • ML/AI: SageMaker*, Elastic Inference, Lex, Polly, Translate, Transcribe, Textract, Rekognition, Ground Truth
  • Business Apps: Chime, WorkDocs, WorkMail

Cloud Architecture Patterns

Decoupling of compute and memory from storage is common in the cloud.

This often involves hybrids of shared-disk parallelism and shared-nothing parallelism.

Dask Task Parallelism Best Practices

Data Partition Sizes

  • Avoid too few chunks (results in a low degree of parallelism).
  • Avoid too many chunks (leads to task graph overhead).
  • Be mindful of available DRAM.

Task Graph Optimization

Use the Diagnostics dashboard to monitor number of tasks, core/node usage, and task completion.

Task Graph sizes:

  • If too large: Indicates bottlenecks (serialization, communication, scheduling).
  • If too small: Indicates under-utilization of cores/nodes.

Rough Guidelines for Optimization

  • Tune data chunk size to adjust the number of tasks.
  • Break up a task or computation.
  • Fuse tasks/computations (known as “batching”), or in other cases, break jobs apart into distinct stages.