In the previous chapters we said the operating system manages the hardware and protects programs from each other. That raises two practical questions:
How does the OS stop a normal program from doing dangerous things, like switching off interrupts or writing over another program's memory?
How is the OS itself organised? Is it one giant program, or many small pieces?
This chapter answers both. First we look at the kernel and the hardware support that protects it (user mode and kernel mode). Then we look at the main ways a kernel can be structured: monolithic, layered, microkernel, modular and hybrid.
What Is a Kernel?
The kernel is the core part of the operating system. It is loaded into memory at boot, stays there all the time, and runs with full control over the hardware. It provides the most basic services: process and CPU management, memory management, device control, file systems, and system call handling.
Everything else, such as your shell, the desktop, the browser and even many system tools, runs outside the kernel as ordinary programs. They must ask the kernel whenever they need something that touches hardware or shared resources.
The main jobs of a kernel:
Kernel job | What it means |
|---|---|
Process management | Create, schedule, switch and terminate processes and threads |
Memory management | Give memory to processes, maintain page tables, keep address spaces separate |
Device management | Talk to hardware through device drivers, handle interrupts |
File systems | Turn disk blocks into files and directories |
System calls | The official entry points programs use to request kernel services |
Protection | Check permissions and keep programs isolated |
Dual-Mode Operation: User Mode vs Kernel Mode
If any program could execute any CPU instruction, then a buggy or malicious program could simply disable interrupts (so the OS never gets control back), reprogram the disk controller, or change the memory mapping to read another program's data. Software rules alone cannot stop this, because the program is running directly on the CPU.
So the protection must come from the hardware. Modern CPUs have (at least) two modes of execution, selected by a mode bit in the CPU:
Feature | User mode | Kernel mode |
|---|---|---|
Also called | Unprivileged mode | Supervisor mode, privileged mode, system mode |
Who runs here | Application programs (browser, editor, games) | The kernel (and drivers inside the kernel) |
Privileged instructions | Not allowed – the CPU raises an exception | Allowed |
Memory access | Only its own address space | All memory |
Effect of a crash | Only that program dies | Can bring down the whole system |
x86 name | Ring 3 | Ring 0 |
On x86 CPUs there are four "rings" (0 to 3), but mainstream operating systems like Linux and Windows use only two: ring 0 for the kernel and ring 3 for user programs. ARM CPUs use "exception levels" (EL0 for user programs, EL1 for the kernel) for the same idea.
Privileged Instructions
A privileged instruction is one that can only be executed in kernel mode. Typical examples:
Enabling or disabling interrupts
Direct I/O instructions to device controllers
Changing the memory management registers (for example, loading a new page table)
Setting the timer
Halting the CPU
Switching the mode bit to kernel mode directly
Ordinary instructions such as adding two numbers, comparing values, or reading your own variables are non-privileged and run in user mode at full speed.
See It Happen
This small C program tries to execute hlt, the x86 instruction that halts the CPU, from a normal user program:
#include <stdio.h>
int main(void) {
printf("About to execute HLT (halt the CPU) from user mode...\n");
fflush(stdout);
__asm__ volatile ("hlt"); /* privileged x86 instruction */
printf("This line is never printed.\n");
return 0;
}Compiled with GCC and run on x86-64 Linux:
About to execute HLT (halt the CPU) from user mode...
Segmentation fault
The program exited with status 139 (128 + signal 11, SIGSEGV), and the kernel log (dmesg) recorded what happened:
traps: privileged[15888] general protection fault ip:6277af99718f sp:7ffe6a6ee150 error:0 in privileged[118f,6277af997000+1000](The PID and addresses in this line will be different on your machine.)
The CPU refused to execute hlt in user mode and raised a general protection fault. That exception switched the CPU into kernel mode, the kernel saw that a user program tried something illegal, and it killed only that program. The computer kept running normally. This is dual-mode protection in action.
How Does the Mode Change?
user mode (mode bit = user)
+-----------------------------------------+
| user program runs |
+-----------------------------------------+
| ^
| system call / interrupt / | return-from-trap
| exception | instruction
| (hardware switches to | (switches back to
| kernel mode automatically) | user mode)
v |
+-----------------------------------------+
| kernel handles the event |
+-----------------------------------------+
kernel mode (mode bit = kernel)
The key point: a user program cannot simply set the mode bit to kernel. The only ways into kernel mode are controlled entry points: a system call (a special trap instruction), a hardware interrupt, or an exception. In all three cases the CPU jumps to a kernel address that the kernel itself set up at boot, so the program cannot choose to jump into the middle of kernel code. How system calls work step by step is the topic of the next chapter, System Calls.
The Timer Completes the Protection
Dual mode stops a program from doing forbidden things, but what about a program that just loops forever doing allowed things? For that, the kernel programs a hardware timer before handing the CPU to a user program. When the timer interrupt fires, the CPU switches to kernel mode, and the kernel decides whether to let the program continue or switch to another one. Setting the timer is itself a privileged instruction, so programs cannot turn it off.
Kernel Space and User Space
Memory is also divided by privilege:
Kernel space: the memory where the kernel's code and data live. Only accessible in kernel mode.
User space: the memory used by user programs. Each process gets its own separate user space.
+---------------------------+ +---------------------------+
| Process A (user space) | | Process B (user space) |
| code, data, heap, stack | | code, data, heap, stack |
+---------------------------+ +---------------------------+
===================== system call boundary ====================
+-------------------------------------------------------------+
| Kernel space (shared) |
| scheduler, memory manager, file systems, drivers, network |
+-------------------------------------------------------------+
| Hardware |
+-------------------------------------------------------------+
Which services live in kernel space and which live in user space is exactly what separates the different kernel designs below.
OS Structures (Kernel Designs)
An OS is a huge program. Linux, for example, has tens of millions of lines of source code. How you divide that code into parts affects speed, reliability, security and how easy it is to maintain. Let us look at the main designs.
1. Simple Structure
Early, small systems such as MS-DOS had no well-defined structure. Application programs could call into the OS at many points and could even access the hardware directly, partly because the Intel 8088 CPU it was built for had no dual-mode protection.
Application programs
| \
v \ (could bypass the OS)
Resident system programs
| \
v v
MS-DOS device drivers
|
v
ROM BIOS device drivers / Hardware
Problem: one bad program could crash the whole computer. There was no protection.
2. Monolithic Kernel
In a monolithic kernel, the entire operating system (scheduler, memory manager, file systems, device drivers, network stack) runs as one large program in kernel space, in a single address space.
+-----------------------------------------------------------+
| User space: shell browser editor compiler ... |
+-----------------------------------------------------------+
| System call interface |
+-----------------------------------------------------------+
| KERNEL SPACE (one big program) |
| CPU scheduler | memory mgmt | file systems | net stack |
| device drivers | IPC | signal handling | ... |
+-----------------------------------------------------------+
| Hardware |
+-----------------------------------------------------------+
Traditional UNIX and Linux are monolithic.
Why is it fast? All kernel parts are in the same address space, so the file system can call the disk driver with an ordinary function call. No messages, no extra mode switches.
Disadvantages:
A bug in any part (for example, one faulty device driver) can crash the entire system, because everything runs in kernel mode.
Very large code base that is hard to understand, test and maintain.
3. Layered Approach
Here the OS is divided into layers, one on top of another. Layer 0 is the hardware, the top layer is the user interface. Each layer uses only the services of the layer directly below it.
Layer N : user interface
...
Layer 3 : I/O management
Layer 2 : device communication
Layer 1 : memory management
Layer 0 : hardware
The classic example is the THE operating system built by Edsger Dijkstra's team in the 1960s.
Advantage: very easy to build and debug. Test layer 1 first; once it works, build and test layer 2 assuming layer 1 is correct, and so on.
Disadvantages:
Hard to decide the order of layers. For example, the memory manager might need to read from the disk (for virtual memory), but the disk driver might also need memory buffers. Who goes below whom?
Slow: a request may have to pass through many layers, each adding overhead.
Pure layered systems are rare today, but the idea of layering appears everywhere (the network protocol stack is the best-known example).
4. Microkernel
A microkernel keeps only the absolute minimum in kernel space, usually low-level memory management (address spaces), basic CPU scheduling, and inter-process communication (IPC). Everything else, such as device drivers, file systems and the network stack, runs as separate user-space server processes.
USER SPACE
+-----------+ +-------------+ +---------+ +----------+
| your app | | file system | | device | | network |
| | | server | | drivers | | server |
+-----------+ +-------------+ +---------+ +----------+
\ | | /
\______________|_______________|___________/
| messages (IPC)
KERNEL SPACE +-----------------------------------+
| microkernel: IPC, basic scheduling,|
| address spaces |
+-----------------------------------+
Hardware
When your app wants to read a file, it sends a message to the file-system server through the microkernel. The file-system server sends a message to the disk-driver server, and replies come back the same way.
Advantages:
Reliability: if a driver crashes, only that server process dies; it can often be restarted while the system keeps running.
Security: a small kernel is easier to verify. The seL4 microkernel has a formal mathematical proof of correctness.
Extensibility: new services are added as user programs, without touching the kernel.
Disadvantage: performance. Every service request becomes messages that cross between user and kernel mode several times. Each crossing and copy costs time. Modern microkernels have reduced this cost a lot, but it is still the classic trade-off.
Examples: Mach (research kernel from Carnegie Mellon), MINIX 3, QNX (widely used in cars and embedded systems), seL4.
5. Modular Kernel (Loadable Kernel Modules)
Most modern kernels use loadable kernel modules. The core kernel is small-ish, and extra parts such as drivers for specific hardware or extra file systems can be loaded into the running kernel when needed and unloaded later, without rebooting or recompiling the kernel.
Linux is the best example. It is monolithic but modular: modules run in kernel space with full privileges (so they are as fast as built-in code), but they can be added and removed dynamically. You can list the currently loaded modules with lsmod. Here is real output from a Linux machine (the first few lines):
lsmod | head -6Module Size Used by
intel_rapl_msr 16384 0
intel_rapl_common 40960 1 intel_rapl_msr
kvm_intel 368640 0
kvm 1003520 1 kvm_intel
irqbypass 16384 1 kvmThe "Used by" column shows dependencies: kvm_intel (Intel virtualization support) uses kvm, so kvm cannot be unloaded while kvm_intel is loaded. Modules are managed with modprobe (load/unload with dependencies), insmod and rmmod, which need root privileges.
This design combines the good parts of the others: the speed of a monolithic kernel and the flexibility of plugging in only what you need. But note: a buggy module still runs in kernel mode and can still crash the system, unlike a microkernel driver.
6. Hybrid Kernel
In practice, most big commercial operating systems mix ideas:
Windows (NT family): often described as a hybrid. It has a layered, microkernel-inspired design, but for performance most services (including the window manager and many drivers) run in kernel mode.
macOS and iOS (XNU kernel): combines the Mach microkernel with BSD Unix components, but runs them together in kernel space for speed, and supports loadable extensions.
Critics say "hybrid" is mostly a monolithic kernel with a microkernel-style internal design. For exams, it is enough to know: hybrid kernels try to get microkernel structure with monolithic performance.
7. Exokernel (just for awareness)
An exokernel is a research design (from MIT) that goes even smaller than a microkernel: the kernel only securely divides hardware resources, and applications manage those resources themselves through library operating systems. It is rarely asked beyond a definition.
Monolithic vs Microkernel: The Big Comparison
Feature | Monolithic kernel | Microkernel |
|---|---|---|
What runs in kernel space | Almost the entire OS | Only IPC, basic scheduling, address-space management |
Drivers and file systems | Inside the kernel | User-space server processes |
Communication between parts | Direct function calls | Message passing (IPC) |
Performance | Faster | Slower (more mode switches and messages) |
Reliability | One faulty driver can crash the system | A faulty driver crashes only its server |
Kernel size | Large | Small |
Extending the OS | Modify / load into the kernel | Add a new user-space server |
Examples | Linux, traditional UNIX, FreeBSD | Mach, MINIX 3, QNX, seL4 |
A famous piece of history: in 1992 Andrew Tanenbaum (author of MINIX) and Linus Torvalds (author of Linux) had a public online debate about exactly this, with Tanenbaum arguing that monolithic kernels were obsolete. Linux stayed monolithic (with modules) and became hugely successful, while microkernels found their place in embedded, safety-critical and high-security systems. Both designs are still alive because they optimise for different things.
Where Do Common Operating Systems Fit?
OS | Kernel | Structure |
|---|---|---|
Linux, Android | Linux | Monolithic with loadable modules |
Windows 10/11 | Windows NT kernel | Hybrid |
macOS, iOS | XNU | Hybrid (Mach + BSD) |
MINIX 3, QNX, seL4 | their own | Microkernel |
MS-DOS | — | Simple structure, no protection |
Common Mistakes
"Kernel mode means the root/administrator user." No. Root is a user account with extra permissions, but programs run by root still execute in user mode. Only kernel code runs in kernel mode. Root just has the right to ask the kernel for more things (like loading a module).
"Linux is a microkernel because it has modules." No. Linux modules run inside kernel space; Linux is monolithic and modular.
"A user program can switch to kernel mode by setting the mode bit." No. Changing the mode is privileged. The only way in is through a trap (system call), interrupt or exception, which jumps to kernel-defined entry points.
"Microkernels are always better because they are safer." It is a trade-off. Safety and modularity are paid for with IPC overhead.
Interview Questions
What is the difference between user mode and kernel mode?
In kernel mode the CPU can execute any instruction and access all memory; the kernel runs here. In user mode, privileged instructions are blocked and a program can only access its own memory; applications run here. A mode bit in the CPU tracks the current mode.
Why do we need dual-mode operation?
To protect the OS and other programs from errant or malicious user programs. Dangerous instructions can be executed only by the kernel, and every entry into kernel mode goes through controlled points.
Give examples of privileged instructions.
Disabling interrupts, I/O instructions, loading page-table/memory-management registers, setting the timer, halting the CPU.
Monolithic vs microkernel?
Monolithic: the whole OS runs in kernel space; fast but a bug anywhere can crash everything (Linux). Microkernel: only minimal services in the kernel, the rest as user-space servers communicating by messages; more reliable and secure but slower (QNX, MINIX 3, seL4).
What is a loadable kernel module?
A piece of kernel code (often a device driver) that can be loaded into or removed from a running kernel without rebooting. It runs in kernel mode. Linux uses them heavily (lsmod, modprobe).
What kind of kernel does Windows / macOS use?
Both are usually described as hybrid kernels: Windows NT kernel and XNU (Mach plus BSD), with most services running in kernel space for performance.
Summary
The kernel is the always-resident, privileged core of the OS.
Dual-mode operation (user mode and kernel mode) is enforced by hardware; privileged instructions in user mode cause an exception.
User programs enter the kernel only through system calls, interrupts and exceptions. A timer guarantees the kernel regains control.
Monolithic: everything in kernel space, fast, less isolated (Linux, UNIX).
Layered: each layer uses only the one below; easy to debug, slow and hard to order.
Microkernel: minimal kernel plus user-space servers via IPC; reliable and secure, slower (QNX, MINIX 3, seL4).
Modular: loadable kernel modules (Linux). Hybrid: a mix (Windows NT, XNU).
Next, in System Calls, we will see exactly how a user program crosses the boundary into the kernel to ask for a service.