A computer that cannot talk to the outside world is useless. Keyboards, screens, disks, SSDs, network cards, USB drives, cameras and printers are all I/O (Input/Output) devices, and managing them is one of the oldest and hardest jobs of an operating system.
The difficulty is variety. A keyboard produces a few bytes per second, an NVMe SSD can move gigabytes per second, a network card receives packets whenever they arrive, and every device family speaks its own hardware language. Yet your program simply calls read() and write() and everything works.
In this chapter we will see how that magic is built: the hardware side (controllers, registers, ports), the three ways of moving data (programmed I/O, interrupts and DMA), the software layers (device drivers and the kernel I/O subsystem), and techniques such as buffering, caching and spooling. We will also measure the effect of buffering with a real C program on Linux.
Why I/O Is a Big Deal for the OS
Three facts make I/O special:
Devices are slow compared to the CPU. A CPU executes billions of instructions per second. Even a fast SSD read takes tens of microseconds, which is tens of thousands of CPU instructions. A keypress may come after seconds. The OS must keep the CPU busy with other work while waiting.
Devices are diverse. Each has different speed, data unit (bytes, blocks, packets), access style (sequential or random) and control commands. The OS must hide this mess behind a uniform interface.
Devices are shared and dangerous. If any program could program the disk controller directly, it could overwrite anyone's files. So I/O instructions are privileged, and all I/O goes through the kernel using system calls (see System Calls).
Goal of the I/O system: give programs a simple, uniform and safe interface to very different devices, while keeping the CPU and the devices as busy as possible.
I/O Hardware: Devices, Controllers and Buses
A device is not connected straight to the CPU. In between sits a device controller (also called an adapter): a small piece of electronics, often with its own processor and memory, that understands the device and talks to the rest of the computer over a bus such as PCI Express (PCIe) or USB.
+-------+ +--------+
| CPU | | Memory |
+---+---+ +---+----+
| |
=========+================+=========== system bus / PCIe
| | |
+------+------+ +------+------+ +------+------+
| Disk | | Network | | USB |
| controller | | controller | | controller |
+------+------+ +------+------+ +---+-----+---+
| | | |
[SSD] [cable] [keyboard][mouse]
The CPU never touches the spinning platter or the flash chips. It only talks to the controller, by reading and writing the controller's registers.
Device Controller Registers
A typical controller exposes four kinds of registers:
Register | Who writes it | Purpose |
|---|---|---|
Data-in | Device | Holds data the device has produced, for the CPU to read |
Data-out | CPU | Holds data the CPU wants to send to the device |
Status | Device | Bits such as busy, data ready, error |
Control (command) | CPU | Tells the device what to do: start, read, write, reset, mode |
How Does the CPU Reach These Registers?
There are two designs.
Port-mapped I/O (isolated I/O) | Memory-mapped I/O |
|---|---|
Device registers live in a separate I/O address space (ports) | Device registers are given addresses in the normal memory address space |
Needs special instructions, for example x86 | Normal load/store instructions ( |
Protection: the special instructions are privileged | Protection: the kernel simply does not map those addresses into user processes' page tables |
Used for legacy PC devices (keyboard controller, serial ports) | Used by almost all modern devices (PCIe cards, graphics memory, NVMe) |
Memory-mapped I/O won because it is simpler for compilers and drivers (any instruction that works on memory works on devices) and because devices like graphics cards need large regions (frame buffers) that fit naturally into the memory map. One catch: such addresses must not be cached by the CPU, otherwise the CPU would read a stale status register. The kernel marks those pages uncacheable, and C driver code uses volatile so the compiler does not optimise the reads away.
Three Ways to Do I/O
Suppose a program wants to read data from a device. Who moves each byte, and how does the CPU know the device is ready? There are three classic answers, each fixing a weakness of the previous one.
1. Programmed I/O (Polling / Busy Waiting)
The CPU does everything itself. For each byte (or word):
Keep reading the status register until the ready bit is 1. This loop is called polling or busy waiting.
Read the byte from the data-in register.
Store it in memory. Repeat.
/* Simplified driver loop for programmed I/O */
while (count > 0) {
while ((*status_reg & READY) == 0)
; /* busy wait: CPU does nothing useful */
*buffer++ = *data_in_reg; /* CPU copies the byte itself */
count--;
}
Problem: while waiting, the CPU is 100% occupied doing nothing useful. For a slow device this wastes millions of instructions.
But polling is not always bad. If the device is very fast and the answer will arrive in a few hundred nanoseconds, a short poll is cheaper than the overhead of an interrupt. That is why Linux network drivers switch to polling under heavy load (the NAPI mechanism) and why high-performance storage stacks can poll NVMe completion queues.
2. Interrupt-Driven I/O
Instead of the CPU asking again and again "are you ready?", the device tells the CPU when it is ready by raising an interrupt.
CPU Device controller
| 1. issue command (read) ------------> |
| 2. go run other processes | ... device works ...
| |
| <------------ 3. interrupt signal ----- | data ready
| 4. save state, jump to handler |
| 5. handler copies data from data-in |
| 6. wake up the waiting process |
| 7. resume what it was doing |
The CPU now does useful work while the device is busy. The cost is the interrupt overhead (saving registers, running the handler, returning) which is paid once per interrupt. If the device interrupts for every single byte, a fast device can drown the CPU in interrupts.
3. Direct Memory Access (DMA)
For bulk data (disk blocks, network packets), even one interrupt per byte is too much. So we add a DMA controller that can move data between the device and main memory without the CPU.
The CPU (driver) programs the DMA controller: source, destination address in memory, byte count, direction.
The CPU goes off to do other work.
The DMA controller transfers the whole block directly into memory, using the bus itself.
When the entire block is done, it raises one interrupt.
+-----+ program DMA (addr, count) +-----+
| CPU | ----------------------------------> | DMA |
+-----+ +--+--+
^ |
| one interrupt at the end | moves the block
+-------------------------------------------+
v
[Device] ======== data ========> [Memory]
Modern devices usually have the DMA engine built into their own controller ("bus mastering"), and they use lists of buffers (scatter-gather DMA) so one operation can fill several non-contiguous memory pages.
Comparison Table
Feature | Programmed I/O | Interrupt-driven I/O | DMA |
|---|---|---|---|
Who moves the data? | CPU | CPU (in the handler) | DMA controller |
How CPU learns device is ready | Polls status register | Interrupt per byte/word | Interrupt per block |
CPU free while waiting? | No | Yes | Yes, and also free during the transfer |
Best for | Very fast, very short waits; simple embedded systems | Slow, character devices (keyboard, mouse) | Bulk transfers (disk, SSD, network, GPU) |
Worked Example: How Much CPU Time Does Each Method Cost?
A device delivers data at 50 KB/s (take 1 KB = 1000 bytes), one byte at a time. Handling one interrupt (save state, run handler, return) costs 2 µs of CPU time.
Time between bytes = 1 / 50,000 s = 20 µs.
(a) Programmed I/O with busy waiting. The CPU sits in the polling loop the whole time, so it is 100% busy with this device.
(b) Interrupt-driven I/O. One interrupt every 20 µs, each costing 2 µs:
CPU fraction = 2 / 20 = 0.10 = 10%
Interrupts per second = 50,000
Gain over busy waiting = 20 / 2 = 10 times less CPU time
(c) DMA. Now the device transfers blocks of 4000 bytes, and the CPU spends 10 µs per block (setting up DMA plus handling the completion interrupt).
Time to fill one block = 4000 x 20 µs = 80,000 µs = 80 ms
CPU fraction = 10 / 80,000 = 0.000125 = 0.0125%
Interrupts per second = 50,000 / 4000 = 12.5
From 100% to 10% to 0.0125%. That is why every serious bulk device uses DMA.
DMA Cycle Stealing
DMA is not completely free. The DMA controller and the CPU share the memory bus, so while DMA uses the bus the CPU may have to wait. In cycle stealing mode, DMA grabs the bus for one memory cycle per word, then gives it back.
Example: a device sends 2 MB/s (2,000,000 bytes/s) in 4-byte words using cycle stealing. Each stolen memory cycle takes 40 ns.
Words per second = 2,000,000 / 4 = 500,000
Time per word = 1 / 500,000 s = 2 µs = 2000 ns
CPU blocked = 40 / 2000 = 2% of the time (worst case)
The three DMA transfer modes you should know:
Burst (block) mode: DMA keeps the bus until the whole block is transferred. Fastest transfer, but the CPU cannot use the bus meanwhile.
Cycle stealing mode: DMA takes the bus for one word at a time. Transfer is slower, CPU is delayed only a little at a time.
Transparent mode: DMA uses the bus only when the CPU is not using it. No CPU slowdown, but the slowest transfer.
Interrupts in More Detail
Interrupts are the heartbeat of an OS, so it is worth knowing how they are handled.
The device controller asserts an interrupt request line. On modern PCs this goes through an interrupt controller (APIC), or the device sends a special memory write called an MSI/MSI-X message.
After finishing the current instruction, the CPU checks for pending interrupts.
The CPU saves the minimum state (program counter, flags), switches to kernel mode, and uses the interrupt number to look up the interrupt vector table (on x86, the IDT with 256 entries; vectors 0 to 31 are reserved for CPU exceptions).
The interrupt service routine (ISR) for that device runs.
The CPU restores state and returns to the interrupted code, or the scheduler picks a different process if the interrupt woke up a more important one.
Important ideas:
Maskable vs non-maskable interrupts. Normal device interrupts can be temporarily disabled (masked) during critical kernel code. A non-maskable interrupt (NMI) is reserved for serious events such as hardware errors and watchdogs.
Interrupt priority. A high-priority interrupt can preempt the handler of a lower-priority one.
Top half and bottom half. The ISR must be short because other interrupts may be blocked while it runs. So Linux splits work: the top half (the real ISR) acknowledges the device and grabs the data quickly; the bottom half (softirqs, tasklets, workqueues) does the slower processing later with interrupts enabled.
Interrupt coalescing. A fast network card may wait until several packets have arrived and then raise one interrupt, trading a tiny bit of latency for much less CPU overhead.
On Linux you can see the interrupt counts per CPU in /proc/interrupts, and the type of each device file in /dev.
The Software Side: Layers of I/O
The I/O software is organised in layers. Each layer hides the details of the one below it.
+-------------------------------------------+
| User program: read(fd, buf, n) | user space
| C library (stdio buffering, printf...) |
+===========================================+ system call boundary
| Device-independent I/O software |
| (VFS, naming, permissions, buffering, | kernel space
| caching, scheduling, error reporting) |
+-------------------------------------------+
| Device drivers (one per device type) |
+-------------------------------------------+
| Interrupt handlers |
+===========================================+
| Hardware: controllers and devices |
+-------------------------------------------+
Device Drivers
A device driver is the kernel code that knows the details of one specific controller. It translates generic requests ("read block 5000") into the exact register writes that controller understands, and handles its interrupts.
The rest of the kernel talks to every driver through a standard interface, so adding a new device means writing a new driver, not changing the whole OS. In Linux, drivers are usually loadable kernel modules (lsmod lists them). Because drivers run in kernel mode, a buggy driver can crash the whole system; studies of Linux and Windows have repeatedly found drivers to be a major source of kernel bugs. This is one argument for microkernels, which run drivers in user space (see OS Structure & Kernel).
Device Categories
The OS groups devices by how they are accessed:
Category | Access style | Examples | Typical interface |
|---|---|---|---|
Block devices | Fixed-size blocks, random access, can seek | HDD, SSD, USB drive |
|
Character devices | Stream of bytes, usually sequential | Keyboard, mouse, serial port, terminal |
|
Network devices | Packets, data arrives on its own schedule | Ethernet, Wi-Fi cards | Sockets ( |
On Linux, block and character devices appear as special files in /dev. The first letter of ls -l tells the type (b or c), and instead of a file size you see the major, minor numbers: the major number selects the driver, the minor number selects which device that driver handles. Real output from WSL2 Ubuntu:
$ ls -l /dev/sda /dev/sdb /dev/null /dev/zero /dev/tty /dev/urandomcrw-rw-rw- 1 root root 1, 3 Nov 2 10:55 /dev/null
brw-rw---- 1 root disk 8, 0 Nov 2 10:55 /dev/sda
brw-rw---- 1 root disk 8, 16 Nov 2 10:55 /dev/sdb
crw-rw-rw- 1 root tty 5, 0 Nov 2 10:55 /dev/tty
crw-rw-rw- 1 root root 1, 9 Nov 2 10:55 /dev/urandom
crw-rw-rw- 1 root root 1, 5 Nov 2 10:55 /dev/zero
Both disks use driver major 8 (SCSI disk), with different minor numbers (each whole disk gets 16 minors, room for its partitions). /dev/null, /dev/zero and /dev/urandom are all handled by the "memory devices" driver (major 1), told apart by their minor numbers 3, 5 and 9. (ls sorts by name, which is why the list is not in the order typed.) This is the Unix idea of "everything is a file": the same open/read/write calls work on files and devices. Devices that do not fit this model use the catch-all ioctl() system call for device-specific commands (for example, setting terminal options).
Blocking, Non-blocking and Asynchronous I/O
From the program's point of view, there are three styles of I/O call:
Style | What the call does | Example |
|---|---|---|
Blocking | Process is put in the waiting state until the data is available. Simplest to program. (Default.) | Plain |
Non-blocking | Returns immediately with whatever is available, possibly nothing (Linux returns -1 with |
|
Asynchronous | Returns immediately; the whole transfer happens in the background and the program is notified when it is complete | Linux |
The difference between the last two is subtle and a favourite interview question: non-blocking says "give me what you have now", asynchronous says "do the whole thing and tell me when it is done".
Here is non-blocking I/O on an empty pipe (see Inter-Process Communication for pipes):
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
int main(void) {
int p[2];
char buf[64];
pipe(p); /* p[0] = read end, p[1] = write end */
/* Make the read end non-blocking */
int flags = fcntl(p[0], F_GETFL);
fcntl(p[0], F_SETFL, flags | O_NONBLOCK);
ssize_t n = read(p[0], buf, sizeof buf); /* pipe is empty right now */
if (n < 0)
printf("read #1: returned %zd, errno = %s\n", n, strerror(errno));
write(p[1], "hello", 5); /* now put some data in */
n = read(p[0], buf, sizeof buf);
printf("read #2: returned %zd bytes: %.*s\n", n, (int)n, buf);
return 0;
}
Output (GCC 15, Linux):
read #1: returned -1, errno = Resource temporarily unavailable
read #2: returned 5 bytes: hello
"Resource temporarily unavailable" is the text for EAGAIN. Without O_NONBLOCK, the first read would have blocked forever, because nobody else would ever write to the pipe.
The Kernel I/O Subsystem
Above the drivers, the kernel provides several device-independent services.
1. I/O Scheduling
The kernel keeps a queue of pending requests per device and may reorder them for efficiency and fairness. For hard disks this is the disk scheduling you studied in Disk Structure & Disk Scheduling. On Linux the block layer offers schedulers such as mq-deadline, bfq, kyber and none (fast NVMe SSDs often use none because reordering helps little).
2. Buffering
A buffer is a memory area that holds data temporarily while it is moved between two parties (device and process, or two devices).
Buffering exists for three reasons:
Speed mismatch. A modem receives slowly but the disk writes in big fast chunks. Collect data in a buffer and write it in one go.
Size mismatch. The network delivers small packets but the application wants a whole message; the disk works in 4 KB blocks but the program writes 10 bytes.
Copy semantics. When you call
write(fd, buf, n)and it returns, you may immediately changebuf. The kernel must have copied your data into its own buffer, so later changes do not corrupt what gets written.
Buffering schemes:
Scheme | Idea |
|---|---|
No buffering | Process waits for each transfer, then processes it. Input and processing never overlap. |
Single buffer | The device fills a kernel buffer while the process works on the previous data; then the data is moved to the process. |
Double buffer (buffer swapping) | Two buffers: the device fills one while the process empties the other, then they swap. |
Circular buffer | More than two buffers in a ring; smooths out bursts (used for keyboard input, audio, network rings). |
Small calculation. Let T = time to input one block (100 µs), M = time to move a block from the system buffer to the process (10 µs), C = time the process computes on a block (60 µs). In steady state, the time per block is approximately:
Scheme | Formula | Time per block |
|---|---|---|
No buffer | T + C | 100 + 60 = 160 µs |
Single buffer | max(C, T) + M | max(60, 100) + 10 = 110 µs |
Double buffer | max(C, T) | max(60, 100) = 100 µs |
With double buffering the process is limited only by the slower of input and computation, because the two fully overlap.
3. Caching
A cache keeps a copy of data in faster storage so that future accesses are fast. The Linux page cache keeps recently used file blocks in RAM; a second cat of a big file is much faster than the first. (That is the "buff/cache" column in free.)
Buffer vs cache: a buffer may hold the only copy of data in transit; a cache always holds a copy of data that lives elsewhere. In practice the same kernel memory often serves both roles.
4. Spooling
Spooling (Simultaneous Peripheral Operations On-Line) means holding the output for a device that can serve only one job at a time, such as a printer, in a disk queue, and sending the jobs to the device one by one.
If two programs printed at the same time without spooling, their pages would be mixed up. With spooling, each program "prints" to a spool file quickly and continues; a printer daemon (for example CUPS on Linux) feeds the files to the printer in order. Spooling is also a classic way to avoid deadlock on non-shareable devices (see Deadlock Prevention & Avoidance (Banker's Algorithm)), because the process never holds the printer itself.
5. Error Handling and Protection
Drivers retry failed operations (a read error on a disk sector may succeed on retry), and system calls report failures through return values and errno (EIO for a hardware I/O error). And because all I/O instructions are privileged and device registers are not mapped into user address spaces, a user program cannot bypass the kernel's permission checks.
Seeing Buffering in Action
This program writes 100,000 bytes to a file twice: first with one write() system call per byte, then with the stdio function putc(), which collects bytes in a user-space buffer and calls write() only when the buffer is full. It counts the actual write system calls by reading syscw from /proc/self/io.
#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <time.h>
#define N 100000 /* bytes to write */
/* Read the "syscw" line (number of write system calls) from /proc/self/io */
static long write_syscalls(void) {
char line[128];
long v = -1;
FILE *f = fopen("/proc/self/io", "r");
while (f && fgets(line, sizeof line, f))
if (sscanf(line, "syscw: %ld", &v) == 1) break;
if (f) fclose(f);
return v;
}
static double now_ms(void) {
struct timespec t;
clock_gettime(CLOCK_MONOTONIC, &t);
return t.tv_sec * 1e3 + t.tv_nsec / 1e6;
}
int main(void) {
/* Version 1: one write() system call per byte */
int fd = open("raw.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
long before = write_syscalls();
double t0 = now_ms();
for (int i = 0; i < N; i++) write(fd, "x", 1);
double t1 = now_ms();
long raw_calls = write_syscalls() - before;
close(fd);
/* Version 2: stdio putc(), which collects bytes in a user-space buffer */
FILE *fp = fopen("buffered.txt", "w");
before = write_syscalls();
double t2 = now_ms();
for (int i = 0; i < N; i++) putc('x', fp);
fflush(fp);
double t3 = now_ms();
long buf_calls = write_syscalls() - before;
fclose(fp);
printf("unbuffered write(): %6ld write syscalls, %7.2f ms\n", raw_calls, t1 - t0);
printf("buffered putc() : %6ld write syscalls, %7.2f ms\n", buf_calls, t3 - t2);
return 0;
}
Compile and run on Linux:
gcc -std=c17 -Wall -Wextra bufio.c -o bufio
./bufioOne real run (WSL2 Ubuntu, GCC 15):
unbuffered write(): 100000 write syscalls, 17.03 ms
buffered putc() : 25 write syscalls, 0.17 ms
The times will differ on your machine and from run to run, but the pattern will not. The syscall counts are exact: glibc chose a 4096-byte buffer (the preferred I/O block size the kernel reports for the file, st_blksize), and 100,000 / 4096 = 24.4, so 25 calls. Buffering cut the work about 100 times, simply by crossing the user/kernel boundary less often. Note that the kernel adds a second level of buffering: those 25 writes went into the page cache, and the data reached the disk later. That is why programs that must not lose data call fsync().
Life Cycle of a Blocking Read Request
Putting everything together, here is what happens when a process calls read() on a file whose data is not in memory:
The process makes the
readsystem call; the CPU switches to kernel mode.The kernel checks the arguments and permissions. If the data is already in the page cache, it copies it to the user buffer and returns immediately.
Otherwise the file system works out which disk blocks are needed, and the block layer queues a request (possibly reordered by the I/O scheduler).
The process is moved to the waiting (blocked) state and the scheduler runs another process.
The device driver programs the controller (and its DMA engine) to read the blocks into a kernel buffer.
The device transfers the data by DMA, then raises an interrupt.
The interrupt handler marks the request complete and moves the waiting process back to the ready queue.
When the process is scheduled again, the kernel copies the data into the user buffer and the system call returns the number of bytes read.
Notice how many earlier chapters meet here: system calls, process states, scheduling, file systems and disk scheduling.
Common Mistakes
"DMA means the CPU is not involved at all." The CPU still sets up the transfer and handles the completion interrupt, and it may be slowed down by bus contention (cycle stealing).
"Polling is always worse than interrupts." For very fast devices or very high event rates, polling avoids interrupt overhead and is used deliberately (Linux NAPI, NVMe polling).
Confusing non-blocking with asynchronous. Non-blocking returns what is available now (maybe nothing); asynchronous starts the full operation and notifies on completion.
Confusing buffer, cache and spool. Buffer: temporary holding area for data in transit. Cache: a faster copy of data stored elsewhere. Spool: a queue of whole jobs for a device that cannot interleave them.
Thinking
write()returning means the data is on disk. It usually only means the data reached the kernel's page cache. Usefsync()for durability.Calling memory-mapped I/O "the same as
mmapof a file". Memory-mapped I/O maps device registers into the address space;mmapof a file maps file pages. Related idea, different thing.
Interview Questions
What is a device controller and what is a device driver?
A controller is hardware that operates the device and exposes registers (data, status, control) to the CPU. A driver is kernel software that knows how to program a specific controller and presents a standard interface to the rest of the OS.
Compare programmed I/O, interrupt-driven I/O and DMA.
Programmed I/O: CPU polls the status register and moves every byte, wasting time while waiting. Interrupt-driven: device interrupts the CPU when ready, CPU still moves each byte in the handler. DMA: a DMA controller moves a whole block directly to memory and interrupts once at the end; best for bulk data.
What is the difference between port-mapped and memory-mapped I/O?
Port-mapped I/O uses a separate address space and special instructions (x86 in/out). Memory-mapped I/O places device registers in the normal memory address space, so ordinary load/store instructions access them.
What is cycle stealing?
A DMA mode in which the DMA controller takes the memory bus for one word (one bus cycle) at a time, briefly delaying the CPU, instead of holding the bus for a whole block.
What is spooling? Give an example.
Holding output for a non-shareable device in a disk queue and feeding jobs to the device one at a time. Print spooling: each program writes its job to a spool file and continues; a print daemon prints them in order.
Why is double buffering faster than single buffering?
With two buffers, the device fills one while the process uses the other, so input and processing overlap fully and the time per block becomes max(T, C) instead of max(T, C) + M.
Block device vs character device?
Block devices transfer fixed-size blocks with random access (disks, SSDs). Character devices transfer a stream of bytes, usually sequentially (keyboard, serial port). In ls -l /dev they show as b and c.
Why are interrupt handlers split into a top half and a bottom half?
The top half runs with some interrupts blocked, so it must be very short: acknowledge the device and save data. The heavier processing is deferred to the bottom half, which runs later with interrupts enabled, keeping the system responsive.
Summary
The CPU talks to devices through controllers and their data, status and control registers, accessed by port-mapped or (mostly today) memory-mapped I/O.
Programmed I/O polls and wastes CPU time; interrupt-driven I/O frees the CPU while waiting; DMA also frees it during the transfer, with one interrupt per block.
In our example, CPU cost fell from 100% (polling) to 10% (interrupts) to 0.0125% (DMA).
I/O software is layered: user libraries, device-independent kernel code, device drivers, interrupt handlers, hardware.
Devices are grouped as block, character and network; calls can be blocking, non-blocking or asynchronous.
The kernel I/O subsystem provides scheduling, buffering, caching, spooling, error handling and protection. Buffering alone cut 100,000 system calls to 25 in our test.
Next, in Protection & Security, we look at how the OS decides who is allowed to use files, devices and memory, and how it defends against attacks.