Chapter 32 of 38

File Systems

Everything you saved today, your code, your photos, your downloaded PDFs, is sitting on a disk or SSD as millions of numbered blocks. The hardware knows nothing about "my_resume.pdf". It only understands requests like "read block 7,340,112".

Somebody has to turn that sea of numbered blocks into named, organised, protected files that survive a reboot. That somebody is the file system.

A file system is the part of the operating system that organises data on storage devices into files and directories, and provides the operations to create, read, write, find and protect them.

So far in this tutorial we have looked at the CPU (processes, scheduling) and memory (paging, virtual memory). This chapter starts the storage part. Here we learn what a file is from the OS's point of view, how programs use files, and how the file system is organised. The next chapters, File Allocation Methods and Directory Structure, look at how blocks are assigned to files and how files are named and grouped.


What Is a File?

A file is a named collection of related information that is stored on secondary storage. To users, it is the smallest unit of storage they can name and save.

Why do we need files at all, when programs already have memory?

  • Persistence: memory is lost when the power goes off. Files survive.

  • Size: storage is far larger than RAM.

  • Sharing: many processes, even ones running at different times, can use the same file.

On UNIX-like systems (Linux, macOS) a file's contents are just a sequence of bytes. The OS does not care whether those bytes are a song, a program or a spreadsheet. The meaning is decided by the programs that read the file. This simple "byte stream" model is one of UNIX's best ideas, and Windows treats regular file data the same way.


File Attributes

Besides its data, every file has metadata, information about the file:

Attribute

Meaning

Name

Human-readable name. Stored in the directory, not inside the file.

Identifier

A unique number used internally (the inode number on UNIX).

Type

Regular file, directory, device, link, etc.

Location

Where the file's blocks are on the device

Size

Current size in bytes (and blocks allocated)

Protection

Owner, group, and who may read, write or execute

Timestamps

Last access, last modification, last metadata change, creation

On UNIX, all of this except the name lives in a structure called the inode (index node). In general OS books it is called the File Control Block (FCB). You can see a file's inode with the stat command. Real output from Linux (ext4) for a 26-byte text file:

stat notes.txt
  File: notes.txt
  size: 26        	Blocks: 8          IO Block: 4096   regular file
Device: 8,48	Inode: 50347       Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/   imran)   Gid: ( 1000/   imran)
Access: 2026-11-02 08:28:34.066612178 +0000
Modify: 2026-11-02 08:28:34.064942157 +0000
Change: 2026-11-02 08:28:34.064942157 +0000
 Birth: 2026-11-02 08:28:34.064942157 +0000

Two interesting details:

  • The file has 26 bytes but uses 8 blocks. stat counts in 512-byte units, so that is 8 × 512 = 4096 bytes, one full 4 KB file-system block. Space is given to files in whole blocks, so the remaining 4070 bytes are wasted. This is internal fragmentation, the same idea you met with paging.

  • Modify is when the data last changed. Change is when the inode (metadata, such as permissions) last changed. They are different things.


File Operations

The OS provides system calls to work with files (see System Calls). The basic ones:

Operation

What the OS does

POSIX call

Create

Find space, make a new inode, add an entry in the directory

open(..., O_CREAT), creat()

Open

Search the directory once, load the inode, return a handle

open()

Read

Copy data from the current position, advance the position

read()

Write

Copy data at the current position, allocate blocks if needed

write()

Reposition (seek)

Change the current position, no I/O needed

lseek()

Close

Release the handle and in-memory state

close()

Delete

Remove the directory entry; free the inode and blocks when nothing refers to them

unlink()

Truncate

Keep attributes, cut the length (often to zero)

truncate(), O_TRUNC

Why Is There an open() Call at All?

Why not just say read("/home/riya/notes.txt", ...) every time? Because finding a file by name is expensive: the OS must walk through every directory on the path, reading each one from disk, and check permissions at each step.

open() does that work once. It returns a small integer, the file descriptor, and later calls use that number to jump straight to the file's in-memory information. It is like getting a token at a bank counter: you show your ID once, then just use the token number.


Open-File Tables

The kernel keeps three levels of in-memory structures for open files:

 Process A                    Kernel (shared by all processes)
 fd table
 +----+                       System-wide              In-memory
 | 0  |→ terminal             open-file table          inode table
 | 1  |→ terminal             +------------------+     +--------------+
 | 2  |→ terminal       +---→ | offset = 10      | --→ | inode 50347  |
 | 3  |-----------------+     | mode = read      |     | size, owner, |
 | 4  |-----------------+     | ref count = 1    |  +→ | block list   |
 +----+                 |     +------------------+  |  +--------------+
                        +---→ | offset = 0       | -+
                              | mode = read      |
                              | ref count = 1    |
                              +------------------+
  • Per-process file descriptor table: maps small integers (0, 1, 2, 3...) to entries in the system-wide table. 0, 1 and 2 are standard input, output and error.

  • System-wide open-file table: one entry per open() call. Holds the current file offset, the access mode and a reference count.

  • In-memory inode table: one entry per file that is open, no matter how many times it is opened. Holds the file's metadata and block locations.

The important consequence: the file offset belongs to the open-file entry, not to the file and not to the descriptor. Two separate open() calls get two independent offsets. But dup(), or a child created by fork(), shares the parent's entry, so they share one offset.

Seeing It in a Real Program

This program writes the alphabet to a file and then reads it in different ways:

#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>

int main(void) {
    /* create (or truncate) a file and write 26 letters */
    int fd = open("notes.txt", O_CREAT | O_WRONLY | O_TRUNC, 0644);
    if (fd == -1) { perror("open"); return 1; }
    printf("open() returned fd = %d\n", fd);
    write(fd, "ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26);
    close(fd);

    struct stat st;
    stat("notes.txt", &st);
    printf("inode = %lu, size = %lld bytes, links = %lu\n",
           (unsigned long)st.st_ino, (long long)st.st_size,
           (unsigned long)st.st_nlink);

    /* sequential access: each read continues where the last stopped */
    char buf[6] = {0};
    int a = open("notes.txt", O_RDONLY);
    read(a, buf, 5); printf("seq read 1: %s\n", buf);
    read(a, buf, 5); printf("seq read 2: %s\n", buf);

    /* direct (random) access: jump to byte 20 */
    lseek(a, 20, SEEK_SET);
    read(a, buf, 5); printf("after lseek(20): %s\n", buf);

    /* a second open() gets its own offset */
    int b = open("notes.txt", O_RDONLY);
    read(b, buf, 5); printf("second open, fd %d: %s\n", b, buf);

    /* dup() shares the SAME open-file entry, so the SAME offset */
    int c = dup(a);
    lseek(a, 0, SEEK_SET);
    read(a, buf, 5); printf("read via fd %d: %s\n", a, buf);
    read(c, buf, 5); printf("read via dup fd %d: %s\n", c, buf);

    close(a); close(b); close(c);
    return 0;
}

Compiled with gcc -std=c17 -Wall -Wextra and run on Linux:

open() returned fd = 3
inode = 50347, size = 26 bytes, links = 1
seq read 1: ABCDE
seq read 2: FGHIJ
after lseek(20): UVWXY
second open, fd 4: ABCDE
read via fd 3: ABCDE
read via dup fd 5: FGHIJ

(The inode number will be different on your machine.) Notice:

  • The first descriptor is 3, because 0, 1, 2 are already taken. The kernel always hands out the lowest free number.

  • The second open() (fd 4) starts reading from the beginning: it has its own offset.

  • Reading through fd 3 moved the offset to 5, so reading through the duplicate fd 5 continued with FGHIJ. Same entry, same offset.

This is exactly why, after fork(), a parent and child writing to the same inherited file do not overwrite each other's output: they share one offset (see Process Creation (fork & exec)).


File Types

How does the OS (or a user) know what kind of file something is? Three common approaches:

  • Extension in the name: .c, .pdf, .exe. Windows relies heavily on extensions to choose which program opens a file. It is only a hint: renaming photo.jpg to photo.txt does not change the bytes.

  • Magic number: the first few bytes of the file identify its format. Every compiled Linux executable (ELF format) starts with the bytes 0x7F 'E' 'L' 'F', PDFs start with %PDF, PNGs with \x89PNG. The file command uses these.

  • Type stored by the OS in the metadata. UNIX stores a small set of types in the inode, shown as the first character of ls -l:

ls -l

Type

Example

-

Regular file

notes.txt, /bin/ls

d

Directory

/home

l

Symbolic link

a shortcut to another path

c

Character device

/dev/null, terminals

b

Block device

disks, e.g. /dev/sda

p

Named pipe (FIFO)

see Inter-Process Communication

s

Socket

local IPC endpoints

This is the famous UNIX idea that "everything is a file": devices, pipes and sockets are all accessed with the same open/read/write/close calls.


File Access Methods

How does a program move through a file's data? There are three classic methods.

1. Sequential Access

Data is processed in order, one record after another, like a cassette tape. Each read advances the position automatically. It is the most common method: compilers reading source code, video players, log processing.

beginning                       current position          end
   |------------------------------------|------------------|
                         read / write → advances position
                         rewind       → back to beginning

2. Direct (Relative) Access

The file is viewed as a numbered sequence of fixed-size records or blocks, and the program can jump to any one of them in any order, like choosing a track on a CD. With fixed-size records the position is a simple calculation:

byte offset of record n = n × record size      (records numbered from 0)

Example: 64-byte student records
record 10 starts at byte 10 × 64 = 640
→ lseek(fd, 640, SEEK_SET); read(fd, buf, 64);

Databases and booking systems need direct access: to fetch seat 237 you should not have to read seats 0 to 236 first. In our program, lseek(a, 20, SEEK_SET) was direct access.

3. Indexed Access

A separate index maps keys to record positions. To find a record, search the small index first, then jump directly to the record. Think of the index at the back of a textbook.

Index (sorted by roll no.)          Data file
+---------+--------+                +-----------------------+
| 1021    | rec 7  | -------------→ | rec 7: 1021, Asha ... |
| 1045    | rec 2  |                |          ...          |
| 1102    | rec 0  |                +-----------------------+
+---------+--------+

For very large files the index itself may be indexed (multi-level index). This is the idea behind database indexes and B+ trees.

Method

How you reach record n

Best for

Sequential

Read records 0 to n-1 first

Streaming, logs, batch processing

Direct

Compute its position and jump

Fixed-size records, databases

Indexed

Look up the key in an index, then jump

Searching by key in large files


How a File System Is Organised

Layered Design

A file system is usually built in layers, each using the one below:

Application programs         open("notes.txt"), read(fd, ...)
        ↓
Logical file system          directories, inodes/FCBs, permissions
        ↓
File-organisation module     file's logical block 3 → disk block 91,204
                             free-space management
        ↓
Basic file system            "read physical block 91,204"; buffer cache
        ↓
I/O control                  device drivers, interrupts
        ↓
Devices                      HDD, SSD, USB drive

Layering means, for example, that the same ext4 code works on an HDD, an SSD or a USB stick; only the driver below changes.

On-Disk Structures

A typical UNIX-style file system (like ext4) divides a partition into regions:

+-------+------------+---------------+-------------+------------------------+
| Boot  | Superblock | Free-space    | Inode table | Data blocks            |
| block |            | bitmaps       |             | (file and directory    |
|       |            |               |             |  contents)             |
+-------+------------+---------------+-------------+------------------------+
  • Boot control block: code needed to boot an OS from this partition (may be empty).

  • Superblock (volume control block): describes the whole file system: block size, total blocks, free block count, number of inodes, and so on. If it is damaged the file system cannot be read, so ext4 keeps backup copies.

  • Free-space information: which blocks and inodes are free (covered in File Allocation Methods).

  • Inode table: one inode per file.

  • Data blocks: the actual contents of files and directories.

Real file systems like ext4 repeat this layout in several block groups across the disk to keep a file's inode close to its data.

The Virtual File System (VFS)

Linux supports many file systems at once: ext4 for /, FAT/exFAT on a USB stick, a network file system, and in-memory ones like /proc. Programs do not need to know which is which because of the Virtual File System layer.

         open / read / write / close
                    ↓
        Virtual File System (VFS)  ← one common interface
        ↓         ↓        ↓        ↓
      ext4      exFAT     NFS     procfs

VFS defines generic objects (superblock, inode, file, directory entry) and each file system plugs in its own functions. That is why cat works the same on /proc/cpuinfo (generated by the kernel on the fly) and on a file on an SSD.

Mounting

Before a file system can be used, it must be mounted: attached at some directory (the mount point) in the existing directory tree.

Before:  /                       After: mount /dev/sdb1 /mnt/usb
         ├── home                       /
         └── mnt                        ├── home
             └── usb  (empty)           └── mnt
                                            └── usb  ← root of the USB file system
                                                ├── photos
                                                └── songs

The OS checks that the device contains a valid file system (reads its superblock) and records the mount in a mount table. Windows does the same thing but usually gives each volume a drive letter (C:, D:) instead of a directory.


Crash Consistency and Journaling

Creating a file changes several structures: the inode bitmap, the inode, the directory, the block bitmap and a data block. If the power fails in the middle, the disk may be left inconsistent (for example, a block marked used but owned by no file).

Older systems ran a checker such as fsck at boot to scan the whole disk, which could take a very long time on big disks. Modern file systems avoid this:

  • Journaling (ext4, NTFS, XFS): first write a description of the change to a special log area (the journal), then apply it. After a crash, the OS replays or discards the few unfinished journal entries instead of checking everything.

  • Copy-on-write (Btrfs, ZFS, APFS): never overwrite live data. Write new versions elsewhere and then switch a pointer in one step. This also makes snapshots cheap.

File system

Common use

Note

FAT32

Old USB drives, memory cards

Simple, very compatible; max file size 4 GB minus 1 byte; no permissions

exFAT

Modern USB drives, SD cards

Removes FAT32's 4 GB file limit

NTFS

Windows

Journaling, permissions (ACLs)

ext4

Default on many Linux distributions

Journaling, extents

XFS, Btrfs

Linux servers

XFS journaling; Btrfs copy-on-write with snapshots

APFS

macOS, iOS

Copy-on-write, snapshots, designed for SSDs


File Locking (in Brief)

When several processes use one file, they may need to coordinate, for the same reasons as in Process Synchronization & Race Conditions.

  • Shared lock: many readers at once (like a reader lock).

  • Exclusive lock: one writer, no one else.

  • Mandatory locking: the OS enforces the lock on every access (Windows typically does this for files opened by another program, which is why you sometimes cannot delete a file that is "in use").

  • Advisory locking: the OS records the lock, but processes must check it voluntarily (the normal case on Linux with flock() and fcntl()).


Common Mistakes

  • Thinking the file name is stored in the inode. On UNIX the name is in the directory entry. The inode has everything else. That is why one file can have several names (see Directory Structure).

  • Thinking the offset is stored in the file. It is stored in the open-file table entry. Two open()s have two offsets; dup() and fork() share one.

  • Believing the extension decides the file type. It is a naming convention. The bytes (magic number) decide the real format.

  • Confusing file size with space used. A 26-byte file still uses a full block.

  • Confusing mtime and ctime. ctime on UNIX means inode change time, not creation time.

  • Forgetting to close files. Each process has a limit on open descriptors (often 1024 by default on Linux); leaking descriptors eventually makes open() fail.


Interview Questions

What is a file system?

The OS component that organises storage into named files and directories, keeps their metadata, manages free space, and provides operations like open, read, write and delete.

What is an inode?

A UNIX on-disk structure holding a file's metadata: type, permissions, owner, size, timestamps, link count and the locations of its data blocks. It does not hold the name.

What is a file descriptor?

A small non-negative integer returned by open(). It indexes the process's descriptor table, which points to an entry in the kernel's system-wide open-file table.

Why do we need open() and close()?

open() does the expensive path lookup and permission check once and caches the file's information in memory. close() releases those resources.

What happens to the file offset after fork()?

The child inherits copies of the parent's descriptors, which point to the same open-file table entries, so parent and child share the same offset.

Sequential vs direct access?

Sequential reads records in order; direct access jumps to any record by computing its position (record number × record size).

What is mounting?

Attaching a file system to a directory (mount point) in the existing tree so its files become accessible under that path.

What is VFS?

A kernel layer that offers one common file interface and dispatches each operation to the correct concrete file system (ext4, exFAT, NFS, procfs...).

What is journaling?

Recording intended metadata (and optionally data) changes in a log before applying them, so after a crash the file system can recover quickly to a consistent state.


The Main Idea

A file system turns numbered disk blocks into named, persistent, protected files.

File          → named sequence of bytes on storage
Inode / FCB   → metadata + block locations (not the name)
open()        → path lookup once, returns a file descriptor
Offset        → lives in the open-file table entry
Access        → sequential, direct, indexed
Layout        → boot block, superblock, free-space info, inodes, data
VFS           → one interface over many file systems
Journaling    → fast recovery after crashes

Programs see names and bytes; the file system quietly maps them to inodes and blocks.

We said the inode records where a file's blocks are. In the next chapter, File Allocation Methods, we look at exactly how the file system decides which blocks a file gets, and how it keeps track of free space.