Chapter 01 of 33

Introduction to System Design

When you write a program for a lab assignment, it runs on one laptop, serves one user (you), and if it crashes you just run it again. Now imagine the same program has to serve 50 million students checking their exam results at 10 AM on result day. It must answer in under a second, it must never show one student's marks to another, and it must keep working even if a server catches fire.

Writing the code is only part of the job. Someone has to decide how many machines to use, where the data lives, what happens when a machine fails, and how the pieces talk to each other. That work is called system design.

This chapter gives you the big picture: what system design is, the vocabulary you will use for the rest of this tutorial, and how a simple one-server app grows step by step into a large-scale system. Every later chapter zooms into one piece of this picture.

Small program:   [ code + data on one laptop ]

Large system:    users → DNS → CDN → load balancer → app servers
                                                       │
                                  cache ←──────────────┼──────────────→ databases
                                                       │
                                     message queues ←──┴──→ file storage

What Is System Design?

System design is the process of defining the architecture, components, data and communication of a software system so that it meets its functional requirements and its quality goals such as scale, speed, reliability and cost.

In simple words: system design answers the question "What pieces do we need, and how do they fit together, so that this product works for real users in the real world?"

Think of building a house. Knowing how to lay bricks (coding) is essential, but an architect still has to decide how many floors, where the pillars go, where the water tank sits, and how people will exit in an emergency. System design is the architect's job for software.

Why should a student care?

  • Placements and interviews. Product companies ask system design questions. Fresher rounds usually focus on low-level design (classes and objects), while experienced roles add high-level design (servers, databases, caches). Knowing the basics early makes you stand out.

  • Better code. Once you know that your API will run on 20 servers behind a load balancer, you stop storing user sessions in a local variable.

  • It connects your subjects. System design uses Computer Networks (HTTP, DNS, TCP), Operating Systems (processes, threads, memory), DBMS (indexes, transactions, replication) and Data Structures (hash tables, trees, queues) together. It is where your whole syllabus meets.


High-Level Design vs Low-Level Design

System design is usually split into two levels.

Aspect

High-Level Design (HLD)

Low-Level Design (LLD)

Question it answers

Which components exist and how do they interact?

How is each component built internally?

Building blocks

Servers, load balancers, databases, caches, queues, CDNs

Classes, interfaces, methods, design patterns, database tables

Typical diagram

Boxes and arrows (architecture diagram)

Class diagram, sequence diagram

Main concerns

Scale, availability, latency, consistency, cost

Readability, extensibility, SOLID principles, correctness

Interview example

"Design a URL shortener for 100 million users"

"Design the classes for a parking lot"

Most of this tutorial is about HLD. The chapters Introduction to Low-Level Design and LLD: Design a Parking Lot cover the LLD side.


Requirements: Functional and Non-Functional

Every design starts with requirements. There are two kinds, and you need both.

Functional requirements describe what the system does: the features a user can see.

Non-functional requirements describe how well it does it: speed, scale, availability, security and so on.

Example: a food-delivery app.

Functional (what)

Non-functional (how well)

Users can search restaurants near them

Search results appear in under 300 ms for most users

Users can place and pay for an order

A payment must never be charged twice

Users can track the delivery partner on a map

Location updates reach the user within a few seconds

Restaurants can update their menu

The app stays up during festival-day traffic spikes

Two systems with the same functional requirements can need completely different designs because of non-functional requirements. A college canteen ordering app and a national food-delivery app have similar features, but one runs fine on a single small server and the other needs hundreds of machines across several data centres.

Non-functional requirements drive the architecture. Features tell you what to build; scale, speed and reliability targets tell you how to build it.


The Qualities We Design For

These words appear in every system design discussion. Each has a dedicated chapter later, so here is just a one-line meaning of each.

Quality

Plain meaning

Typical question

Scalability

The system can handle more load by adding resources

What if users grow 10×?

Latency

How long one request takes

How fast does the page load?

Throughput

How many requests are handled per second

How many orders per second can we process?

Availability

Fraction of time the system is up and answering

Is the app reachable at 2 AM on Diwali?

Reliability / durability

The system behaves correctly and does not lose data

If a disk dies, are uploaded photos lost?

Consistency

All users see the same, up-to-date data

After I change my name, does everyone see the new one?

Maintainability

Easy for engineers to understand, change and operate

Can a new team member ship a fix safely?

Security

Only the right people can see and change data

Can a user read someone else's messages?

Cost

Money spent on servers, storage, bandwidth and people

Do we need 100 servers, or would 10 do?

These qualities pull against each other. Making data perfectly consistent across continents adds latency. Making a system highly available needs extra machines, which adds cost. A large part of system design is choosing which quality matters most for this product. We study the most important trade-off formally in the CAP Theorem & PACELC chapter.


How a System Grows: From One Server to Many

The best way to understand the building blocks is to watch a real product grow. Suppose you launch "NotesHub", a site where students share study notes.

Stage 1: Everything on one server

  Users ──→ [ One server: web app + database + files ]

Your code, your database and uploaded PDFs all live on one machine. This is the right choice at launch: cheap, simple, easy to debug. But it has two problems. If the machine goes down, the whole site is down (a single point of failure). And the app and database fight for the same CPU and memory.

Stage 2: Separate the database

  Users ──→ [ App server ] ──→ [ Database server ]

Now each machine can be sized for its own job. The database gets fast disks and lots of RAM; the app server gets CPU.

Stage 3: More app servers behind a load balancer

                         ┌──→ [ App server 1 ] ──┐
  Users ──→ [ Load    ] ─┼──→ [ App server 2 ] ──┼──→ [ Database ]
            [ balancer]  └──→ [ App server 3 ] ──┘

When one app server is not enough, you add more and put a load balancer in front to spread requests. If one app server dies, the others keep serving. This is horizontal scaling, explained in the Scalability: Vertical vs Horizontal chapter.

Stage 4: Cache and CDN

  Users ──→ [ CDN ] (static files: CSS, JS, PDFs, images)
    │
    └────→ [ Load balancer ] ──→ [ App servers ] ──→ [ Cache ] ──→ [ Database ]

Most students read the same popular notes again and again. A cache (an in-memory store such as Redis) keeps hot data in RAM so the database is not hit every time. A CDN serves files from servers close to the user, so a student in Chennai downloads the PDF from a nearby location instead of a far-away data centre.

Stage 5: Replicas, queues and object storage

                                         ┌──→ [ Cache ]
  [ Load balancer ] ──→ [ App servers ] ─┼──→ [ Primary DB ] ──copies──→ [ Read replicas ]
                                         ├──→ [ Message queue ] ──→ [ Workers: thumbnails, emails ]
                                         └──→ [ Object storage: all uploaded PDFs ]
  • Read replicas copy the database so reads are spread over many machines.

  • A message queue lets slow work (making thumbnails, sending emails) happen in the background, so the user gets a fast reply.

  • Object storage holds large files cheaply and durably, outside the database.

Stage 6: Sharding, multiple regions, microservices

When even the primary database cannot handle all the writes, the data is split across many databases (sharding). Very large products also run in several geographic regions and split the code into independent services. Each step adds power but also complexity, so you take it only when you need it.

Key lesson: good design is not "use every component". It is "use the simplest design that meets today's requirements, and know what you will add when the load grows".


The Building Blocks at a Glance

Here is the toolbox you will learn in this tutorial, with the chapter that covers each one.

Building block

What it does

Chapter

Client and server, APIs

How programs ask each other for work

Client-Server Architecture & APIs

Load balancer

Spreads requests across many servers

Load Balancing

Cache

Keeps hot data in fast memory

Caching; Cache Strategies & Eviction Policies

CDN

Serves content from locations near users

Content Delivery Networks (CDN)

Databases

Store data permanently; SQL or NoSQL

SQL vs NoSQL Databases

Replication and sharding

Copy data for safety; split data for scale

Database Replication; Database Sharding & Partitioning

Message queue

Passes work between services asynchronously

Message Queues & Pub/Sub

Object storage

Stores files, images and videos

Object Storage & File Storage

API gateway, rate limiter

Single entry point; protects against overload and abuse

API Gateway & Service Discovery; Rate Limiting

Monitoring

Tells you what is happening inside the system

Monitoring, Logging & Observability

The last part of the tutorial puts these blocks together in full case studies: a URL shortener, a chat application, a news feed, a video streaming service and a ride-sharing service.


Thinking in Trade-offs

There is rarely a single "correct" design. Instead you choose between options, each with a cost. Some trade-offs you will meet again and again:

Choice

You gain

You pay

Add a cache

Faster reads, less database load

Data may be slightly stale; one more thing to run

Keep extra copies of data

Survives machine failures; more read capacity

Copies can disagree for a short time; more storage

Process work in a background queue

Fast response to the user; absorbs spikes

Work finishes later; harder to debug

Split into microservices

Teams deploy independently

Network calls, partial failures, more operations work

Strong consistency everywhere

Everyone always sees the latest data

Higher latency; may be unavailable during network problems

In an interview, the interviewer cares less about which option you pick and more about whether you can say what you gain and what you pay. The next chapter, How to Approach a System Design Interview, shows how to present these choices.


Common Mistakes

  • Starting with technology names. "We'll use Kafka and Cassandra" before knowing the requirements is a red flag. Requirements first, tools later.

  • Ignoring non-functional requirements. A design that has every feature but falls over at 1,000 users is not a design.

  • Over-engineering. A college project with 200 users does not need sharding, microservices and three regions.

  • Thinking there is one right answer. System design is about justified trade-offs, not a memorised diagram.

  • Forgetting failures. Machines, disks and networks fail all the time at scale. Always ask "what happens if this box dies?"

  • Mixing up HLD and LLD. Drawing class diagrams when asked to design a large service (or vice versa) wastes interview time.


Interview Questions

Q1. What is the difference between functional and non-functional requirements? Give an example of each for a chat app.

Functional requirements are features: "users can send one-to-one messages", "users can see when a message is read". Non-functional requirements are quality targets: "messages are delivered within a second for most users", "no message is ever lost", "the service is available 99.99% of the time".

Q2. What is a single point of failure, and how do you remove one?

A single point of failure (SPOF) is a component whose failure brings down the whole system, such as the only database server. You remove it with redundancy: run more than one copy (several app servers behind a load balancer, a standby database replica) and make sure traffic can switch to a healthy copy automatically.

Q3. HLD or LLD: "Design the classes for a library management system"?

LLD. The question is about classes, their relationships and methods (Book, Member, Loan, and so on), not about servers or databases at scale.

Q4. Why not start every project with a fully distributed architecture to be "future-proof"?

Because every component adds cost, latency (extra network hops), failure modes and operational work. A simple design is faster to build and easier to run. Good engineers design so the system can grow (for example, keeping servers stateless) without paying for scale they do not need yet.

Q5. Why might two apps with the same features have very different architectures?

Their non-functional requirements differ: number of users, peak traffic, latency targets, data size, availability needs and budget. These numbers, not the feature list, decide whether one server is enough or hundreds are needed.


Summary

  • System design decides the components, data and communication of a system so it meets both its features and its quality goals.

  • HLD is about servers, databases, caches and how they interact; LLD is about classes and code structure.

  • Functional requirements say what the system does; non-functional requirements (scale, latency, availability, consistency, cost) decide how it is built.

  • Systems grow in steps: one server → separate database → load-balanced app servers → cache and CDN → replicas, queues, object storage → sharding and multiple regions.

  • Every choice is a trade-off. Say what you gain and what you pay.

Start simple, find the bottleneck, fix it, repeat. That is system design in one line.