I/O Systems and Storage
Applications → system calls → VFS / sockets → buffer/page cache → block layer with I/O schedulers (Linux: none, mq-deadline, kyber, bfq) → device driver → controller → device.
Concept
How the CPU Talks to Devices Device registers: status, control and data registers, reached through port I/O (x86 in/out) or memory-mapped I/O (MMIO — the norm on ARM, RISC-V and microcontrollers: registers appear at physical addresses, accessed through volatile pointers).
Polling: the CPU repeatedly reads a status register. Simple, low latency, wastes CPU.
Interrupt-driven I/O: the device interrupts when ready (Module 12).
DMA (direct memory access): a DMA controller copies whole buffers between device and memory without the CPU, and interrupts once at the end. Essential for disks, networks, audio, and CAN/Ethernet controllers moving many frames.
Device Drivers and the I/O Stack Applications → system calls → VFS / sockets → buffer/page cache → block layer with I/O schedulers (Linux: none, mq-deadline, kyber, bfq) → device driver → controller → device. Unix exposes devices as files: character devices (byte streams: terminals, serial ports, /dev/null) and block devices (disks), identified by major/minor numbers.
SSD (NAND flash) Seek (~5 ms) + rotational latency (~4 ms at 7200 rpm) + transfer ~50–100 µs, no moving parts
Fast; NVMe supports many parallel queues Head movement → disk scheduling
Erase blocks, wear leveling, write amplification, TRIM, managed by the flash translation layer (FTL)
Disk scheduling (HDD): FCFS, SSTF (shortest seek first — may starve far requests), SCAN (elevator), C-SCAN (uniform wait times), LOOK/C-LOOK (only as far as the last request). SSDs don't need seek optimization, which is why Linux often uses none for them.
RAID combines disks: RAID 0 (striping, no redundancy), RAID 1 (mirroring), RAID 5/6 (striping with parity), RAID 10 (mirror + stripe).
Every layer buffers: stdio buffers in user space, the kernel's page cache, the device's own cache. Buffering turns many small operations into few large ones — the biggest single I/O optimization.
sequenceDiagram
participant CPU
participant Driver
participant DMA as DMA engine
participant Dev as Device
CPU->>Driver: read(fd, buf, 4096)
Driver->>Dev: program registers:<br/>buffer address, length, start
Driver-->>CPU: thread sleeps,<br/>CPU runs other work
Dev->>DMA: data
DMA->>DMA: copy into memory
Dev-->>CPU: interrupt:<br/>transfer complete
CPU->>Driver: handler wakes<br/>the thread
Driver-->>CPU: read returns 4096Why It Matters
Embedded developers write drivers with MMIO, interrupts and DMA; backend developers tune I/O schedulers, buffering and fsync; everyone benefits from knowing why many small writes are slow.
Syntax
Block devices (ROTA = rotational) Device utilization and latency (sysstat package)
volatile uint32_t *const CAN_STATUS = (uint32_t *)0x40006404; /* memory-mapped register (microcontroller) */
while ((*CAN_STATUS & TX_READY) == 0) { } /* polling */
setvbuf(f, NULL, _IOFBF, 1 << 16); /* larger stdio buffer */Example
23.1 Disk scheduling algorithms 23.2 Buffering: the cost of one system call per byte 23.3 Devices as the kernel sees them
"""Disk scheduling: total head movement for FCFS, SSTF, SCAN, C-SCAN and LOOK (cylinders 0-199)."""
requests = [98, 183, 37, 122, 14, 124, 65, 67]
start, top = 53, 199
def fcfs(reqs, head):
return list(reqs)
def sstf(reqs, head):
left, order = list(reqs), []
while left:
nxt = min(left, key=lambda r: abs(r - head))
order.append(nxt); left.remove(nxt); head = nxt
return order
def scan(reqs, head): # elevator: go down to cylinder 0, then up
down = sorted([r for r in reqs if r <= head], reverse=True)
up = sorted(r for r in reqs if r > head)
return down + [0] + up
def cscan(reqs, head): # go up to the end, jump to 0, continue up
up = sorted(r for r in reqs if r >= head)
low = sorted(r for r in reqs if r < head)
return up + [top, 0] + low
def look(reqs, head): # like SCAN but only as far as the last request
down = sorted([r for r in reqs if r <= head], reverse=True)
up = sorted(r for r in reqs if r > head)
return down + up
print(f"head at {start}, queue {requests}")
for name, fn in [("FCFS", fcfs), ("SSTF", sstf), ("SCAN (down first)", scan), (
…#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <time.h>
static double ms(struct timespec a, struct timespec b) { return (b.tv_sec - a.tv_sec) * 1e3 + (b.tv_nsec - a.tv_nsec) / 1e6; }
int main(void)
{
enum { N = 200000 };
struct timespec t0, t1, t2, t3;
clock_gettime(CLOCK_MONOTONIC, &t0);
int fd = open("/tmp/eh_raw.bin", O_WRONLY | O_CREAT | O_TRUNC, 0644);
for (int i = 0; i < N; i++) write(fd, "x", 1); /* one system call per byte */
close(fd);
clock_gettime(CLOCK_MONOTONIC, &t1);
FILE *f = fopen("/tmp/eh_buf.bin", "w");
for (int i = 0; i < N; i++) fputc('x', f); /* stdio collects bytes in a 4 KB buffer */
fclose(f);
clock_gettime(CLOCK_MONOTONIC, &t2);
char block[N];
for (int i = 0; i < N; i++) block[i] = 'x';
fd = open("/tmp/eh_blk.bin", O_WRONLY | O_CREAT | O_TRUNC, 0644);
write(fd, block, N); /* one system call in total */
close(fd);
clock_gettime(CLOCK_MONOTONIC, &t3);
printf("200 000 x write(1 byte) : %8.1f ms\n", ms(t0, t1));
printf("200 000 x fputc (stdio) : %8.1f ms\n", ms(t1, t2));
printf("1 x write(200
…$ lsblk -d -o NAME,ROTA,SIZE,TYPE | head -4
NAME ROTA SIZE TYPE
sda 1 356.9M disk
sdb 1 159.4M disk
sdc 1 1G disk
$ cat /sys/block/sda/queue/scheduler 2>/dev/null || ls /sys/block | head -3
[none] mq-deadline kyber
$ ls -l /dev/null /dev/zero /dev/urandom | awk '{print $1, $5, $6, $NF}'
crw-rw-rw- 1, 3 /dev/null
crw-rw-rw- 1, 9 /dev/urandom
crw-rw-rw- 1, 5 /dev/zero
$ head -c 8 /dev/urandom | od -An -tx1 | wc -w
8Expected Result
For the classic request queue, FCFS moves the head 640 cylinders; SSTF and SCAN 236; LOOK 208; C-SCAN 183 (not counting the fast return sweep from 199 to 0).
Writing 200 000 bytes with one system call each took hundreds of times longer than stdio's buffered fputc (which issues a system call only every 4 KB) or a single large write. (Absolute times vary.)
The disks are the virtual disks of the WSL2 virtual machine (reported as rotational even though they live on an SSD); the scheduler none is selected. /dev/null, /dev/zero and /dev/urandom are character devices (c) of major number 1 with different minor numbers.
head at 53, queue [98, 183, 37, 122, 14, 124, 65, 67]
FCFS 53 -> 98 -> 183 -> 37 -> 122 -> 14 -> 124 -> 65 -> 67
head movement 640 cylinders
SSTF 53 -> 65 -> 67 -> 37 -> 14 -> 98 -> 122 -> 124 -> 183
head movement 236 cylinders
SCAN (down first) 53 -> 37 -> 14 -> 0 -> 65 -> 67 -> 98 -> 122 -> 124 -> 183
head movement 236 cylinders
C-SCAN (up) 53 -> 65 -> 67 -> 98 -> 122 -> 124 -> 183 -> 199 -> 0 -> 14 -> 37
head movement 183 cylinders
LOOK (down first) 53 -> 37 -> 14 -> 65 -> 67 -> 98 -> 122 -> 124 -> 183
head movement 208 cylinders200 000 x write(1 byte) : 190.8 ms
200 000 x fputc (stdio) : 0.6 ms
1 x write(200 000 bytes): 0.2 ms$ lsblk -d -o NAME,ROTA,SIZE,TYPE | head -4
NAME ROTA SIZE TYPE
sda 1 356.9M disk
sdb 1 159.4M disk
sdc 1 1G disk
$ cat /sys/block/sda/queue/scheduler 2>/dev/null || ls /sys/block | head -3
[none] mq-deadline kyber
$ ls -l /dev/null /dev/zero /dev/urandom | awk '{print $1, $5, $6, $NF}'
crw-rw-rw- 1, 3 /dev/null
crw-rw-rw- 1, 9 /dev/urandom
crw-rw-rw- 1, 5 /dev/zero
$ head -c 8 /dev/urandom | od -An -tx1 | wc -w
8Common Mistakes
Accessing MMIO registers without volatile (the compiler may cache or reorder the accesses).
Busy-polling on a device that could interrupt, burning CPU and power.
Forgetting cache coherency with DMA on systems without coherent DMA (clean/invalidate caches around transfers).
Unbuffered small writes to files or sockets.
Best Practices
Use DMA for bulk transfers, interrupts for events, polling only for very short or very frequent waits.
Batch I/O; size buffers to match the device (4 KB+ for files, MTU-aware for networks).
Leave the I/O scheduler at none/mq-deadline for SSDs; measure before tuning.
On flash-based embedded systems, minimize writes and use wear-leveling file systems.
Interview Relevance
High (embedded especially): "polling vs interrupts vs DMA?", "what is memory-mapped I/O?", "block vs character device?", "compare disk scheduling algorithms", "HDD vs SSD?", "what is RAID 5?", "why is buffered I/O faster?".
Practice Questions
Compute seek + rotational + transfer time for a 4 KB read on a 7200 rpm disk with 8 ms seek and 150 MB/s transfer.
Implement C-LOOK in the simulator.
Write a program that copies a file with buffer sizes 1, 64, 4096 and 65536 bytes and times each.
Sketch the register-level steps a UART driver performs to send one byte with interrupts.
Quick Quiz
Q1. (Multiple choice) Which technique moves a whole buffer between device and memory without the CPU copying each byte? A) Polling B) Interrupts C) DMA D) Spooling Answer: C.
Q2. (Predict the output) Total head movement for SSTF in the example?
Answer: 236 cylinders (verified).
Q3. (True/False) SSTF can starve requests far from the head.
Answer: True. Q4. (Find the bug) uint32_t *status = (uint32_t *)0x40006404; while (!(*status & 1)); hangs even though the flag is set.
Answer: Without volatile, the compiler may read the register once and loop forever on the cached value.
Coding Challenge
Compute the average access time for a 4 KB read from a 7200 rpm disk with an 8 ms average seek and a 150 MB/s transfer rate, printing seek, rotational latency (half a revolution) and transfer components.
#include <stdio.h>
int main(void)
{
double seek_ms = 8.0;
double rotation_ms = 60000.0 / 7200.0; /* one revolution */
double rotational_ms = rotation_ms / 2.0; /* on average half a revolution */
double transfer_ms = 4096.0 / (150e6) * 1000.0;
printf("seek %.2f ms + rotational %.2f ms + transfer %.3f ms = %.2f ms\n",
seek_ms, rotational_ms, transfer_ms, seek_ms + rotational_ms + transfer_ms);
return 0;
}