Open any app on your phone and tap a button. Somewhere, a program you cannot see receives a message, does some work and sends back an answer. The app on your phone is the client, the program far away is the server, and the agreed format of their messages is the API.
Almost every system you will design in this tutorial is built from this one idea, repeated many times: browsers talk to web servers, web servers talk to databases, services talk to other services. In this chapter we look at the client-server model from a system designer's point of view: how a request travels, how systems are split into tiers, what an API contract is, how HTTP methods and status codes are used, and how a server can push updates to clients. We finish by building and running a tiny API in Python.
Client Server
(browser, app) (always running)
│ ─── request: "give me note 7" ───→ │
│ │ does the work
│ ←── response: 200 OK + the note ── │
The Client-Server Model
In the client-server model, a server is a program that offers a service and waits for requests; a client is a program that sends requests to the server and uses its responses.
A few things follow from this definition:
Roles, not machines. "Server" means the program's role. Your laptop can run a server; a big data-centre machine can act as a client of a database.
The client starts the conversation. In classic request-response, the server never calls the client out of nowhere. (We will see ways around this later in this chapter.)
Centralised data and rules. The server owns the data and enforces the rules. A client can be modified by anyone, so the server must never trust the client: validate every input and check every permission on the server side.
One server, many clients. This is why servers must handle many requests at the same time and why scaling servers is the central topic of system design.
The alternative is peer-to-peer (P2P), where every node is both client and server (BitTorrent is the classic example). P2P has no central bottleneck, but it is hard to control, secure and keep consistent. Most products you will design are client-server, often with many servers behind the scenes.
Feature | Client-Server | Peer-to-Peer |
|---|---|---|
Who holds the data | Central servers | Spread across peers |
Control and security | Easy to enforce at the server | Hard; peers are untrusted |
Scaling | Add server capacity as users grow | Each new peer also adds capacity |
Failure | Server down = service down (unless replicated) | No single point of failure |
Examples | Websites, banking, email, most apps | BitTorrent, some blockchain networks |
The Life of a Web Request
When a user opens https://example.com/notes/7, several steps happen before the server code even runs. You studied these protocols in Computer Networks; here is how they look to a system designer.
1. DNS lookup example.com → 203.0.113.10 (cached after the first time)
2. TCP connection 3-way handshake with the server (1 round trip)
3. TLS handshake agree on encryption keys (1 more round trip with TLS 1.3)
4. HTTP request GET /notes/7 ──→ server
5. Server work check auth, read cache/database, build the response
6. HTTP response 200 OK + JSON body ←── server
7. Client renders the note appears on screen
Why this matters for design:
Steps 2 and 3 cost round trips. If the server is far away, each round trip may cost tens or hundreds of milliseconds. That is why connections are reused (HTTP keep-alive, HTTP/2 multiplexing) and why content is moved closer to users with CDNs.
Step 5 is where the server's own design matters: how many database queries, whether a cache is hit, whether slow work is pushed to a background queue.
With HTTP/3, the transport is QUIC over UDP, which combines the transport and TLS handshakes to save a round trip on new connections.
Tiers: Splitting the System
A tier is a layer of the system that can run on its own machines. Splitting into tiers lets each part scale and fail independently.
1-tier: [ UI + logic + data ] e.g. a desktop app with a local file
2-tier: [ Client: UI + logic ] ──→ [ Database ] e.g. old desktop apps talking to a DB
3-tier: [ Client: UI ] ──→ [ App server: logic ] ──→ [ Database: data ]
(presentation) (application) (data)
Architecture | Pros | Cons |
|---|---|---|
1-tier | Simplest; no network | No sharing between users |
2-tier | Shared data; easy to build | Every client connects to the database directly: poor security, hard to change business rules, many DB connections |
3-tier / n-tier | Business rules live in one place; database is hidden; each tier scales separately | More network hops and more components to run |
Almost all modern web systems are n-tier: the middle tier is often split further into a load balancer, several services, caches and queues. The rest of this tutorial is about how to design that middle and data tier.
What Is an API?
An API (Application Programming Interface) is a contract that defines how one piece of software can request work from another: which operations exist, what inputs they take, what outputs and errors they return.
An API hides the how and exposes only the what. The mobile app does not know whether notes are stored in MySQL or MongoDB; it only knows that GET /notes/7 returns a note. That separation is powerful: the server team can change the database, add a cache or split the service, and as long as the contract stays the same, no client breaks.
In system design, "API" usually means a web API: operations exposed over the network, most often using HTTP and JSON. Popular styles are REST, GraphQL and gRPC; they are compared in detail in the API Design: REST, GraphQL & gRPC chapter. Here we focus on the HTTP basics that all of them build on.
Anatomy of an HTTP request and response
POST /notes HTTP/1.1 ← method, path, version
Host: api.example.com ← headers (metadata)
Content-Type: application/json
Authorization: Bearer eyJhbGciOi...
Content-Length: 21
← blank line
{"text": "Revise OS"} ← body
HTTP/1.1 201 Created ← status code and reason
Content-Type: application/json
Location: /notes/1
Content-Length: 30
{"id": 1, "text": "Revise OS"}
HTTP methods
The method tells the server what kind of action the client wants. The current HTTP specification (RFC 9110) defines two important properties:
Safe: the method is essentially read-only; the client does not request any change on the server.
Idempotent: sending the same request several times has the same intended effect on the server as sending it once.
Method | Typical use | Safe? | Idempotent? |
|---|---|---|---|
GET | Read a resource | Yes | Yes |
HEAD | Like GET but headers only | Yes | Yes |
POST | Create something or run an action | No | No |
PUT | Create or fully replace a resource at a known URL | No | Yes |
PATCH | Partially update a resource (defined in RFC 5789) | No | Not guaranteed |
DELETE | Remove a resource | No | Yes |
OPTIONS | Ask which methods/options are supported (used by CORS preflight) | Yes | Yes |
Why idempotency matters so much in distributed systems: suppose a client sends a request and the network drops the response. The client does not know whether the server did the work, so it retries. Retrying PUT /users/5/name is harmless: the name is just set again. Retrying POST /payments could charge the user twice. That is why payment APIs commonly accept an idempotency key: the client sends a unique ID (for example in an Idempotency-Key header) and the server remembers it, so a repeated request returns the first result instead of charging again. (An Idempotency-Key header is being standardised by the IETF; at the time of writing it is still an Internet-Draft, not an RFC.)
Status codes
The three-digit status code tells the client what happened. The first digit gives the class.
Code | Meaning | When to use it |
|---|---|---|
200 OK | Success | A normal successful GET or action |
201 Created | A new resource was created | After POST creates a note; often with a |
204 No Content | Success, no body | After a successful DELETE |
301 / 308 | Moved permanently | Old URL redirects to a new one (308 keeps the method) |
304 Not Modified | Your cached copy is still valid | Conditional GET with |
400 Bad Request | The request is malformed | Missing or invalid field |
401 Unauthorized | Not authenticated | No valid login token |
403 Forbidden | Authenticated but not allowed | A student trying to edit marks |
404 Not Found | No such resource | Unknown note ID |
409 Conflict | Request conflicts with current state | Username already taken; version mismatch |
429 Too Many Requests | Client is sending too fast (RFC 6585) | Rate limit exceeded; see the Rate Limiting chapter |
500 Internal Server Error | Bug or unexpected failure on the server | Unhandled exception |
502 Bad Gateway | A proxy got a bad response from the upstream server | Load balancer cannot get a valid reply from an app server |
503 Service Unavailable | Temporarily overloaded or down | Maintenance or overload; may include |
504 Gateway Timeout | A proxy timed out waiting for the upstream | App server too slow to answer the load balancer |
Rule of thumb: 4xx means "the client should change something"; retrying the same request will not help (except 429, after waiting). 5xx means "the server had a problem"; a retry later may succeed, but only retry idempotent requests or requests protected by an idempotency key.
Stateless vs Stateful Servers
HTTP itself is stateless: each request carries everything needed to understand it, and the protocol does not remember earlier requests. Applications still need state ("Asha is logged in", "her cart has 2 items"). The question is where that state lives.
Stateful server | Stateless server |
|---|---|
Keeps session data in its own memory | Keeps no per-user data between requests |
The next request must reach the same server | Any server can handle any request |
If the server dies, its sessions are lost | A server can die or be added at any time |
Harder to scale out | Easy to scale out behind a load balancer |
To make app servers stateless, the state moves elsewhere: into a token the client sends with every request (such as a signed JWT, covered in Authentication & Authorization: OAuth & JWT), or into a shared store such as Redis or the database. The next chapter, Scalability: Vertical vs Horizontal, shows with running code why this matters.
How Servers Send Updates to Clients
Classic request-response has a limitation: the server can only answer, never start a conversation. But chat apps, live scores and stock tickers need the server to tell the client "something changed". There are four common patterns.
Short polling: C: any news? → S: no C: any news? → S: no C: any news? → S: yes!
Long polling: C: any news? → S: (holds the request open ... ) → S: yes! → C: any news? ...
SSE: C: subscribe → S: event → S: event → S: event ... (one-way, server→client)
WebSocket: C ⇄ S both sides send messages at any time over one connection
Pattern | How it works | Good for | Drawbacks |
|---|---|---|---|
Short polling | Client asks every few seconds | Simple dashboards; works with plain HTTP caches and CDNs | Many empty responses; updates delayed up to one interval |
Long polling | Server holds the request until there is news or a timeout, then the client immediately asks again | Near-real-time where WebSockets are blocked | Server keeps many waiting requests; reconnect after every message |
Server-Sent Events (SSE) | One long HTTP response streams events ( | Live scores, notifications, streaming text: server-to-client only | One direction only; client sends via normal requests |
WebSocket (RFC 6455) | Starts as an HTTP request, then upgrades to a persistent, full-duplex connection | Chat, multiplayer games, collaborative editing | Long-lived connections are stateful: harder to load balance and scale |
We will use these patterns in the case studies, especially Design a Chat Application.
Hands-On: A Tiny Notes API in Python
Let us build a real client and server using only Python's standard library. The server keeps notes in memory and supports POST /notes, GET /notes and GET /notes/{id}. The same script then acts as a client and calls the server, so you can see the status codes from the tables above in action.
import json
import threading
import urllib.request
import urllib.error
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
notes = {} # the server's state: id -> note
next_id = 1
lock = threading.Lock() # ThreadingHTTPServer handles requests in parallel
class NotesAPI(BaseHTTPRequestHandler):
def send_json(self, status, body):
data = json.dumps(body).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self):
if self.path == "/notes":
self.send_json(200, list(notes.values()))
elif self.path.startswith("/notes/"):
note = notes.get(int(self.path.split("/")[-1]))
if note:
self.send_json(200, note)
else:
self.send_json(404, {"error": "note not found"})
else:
self.send_json(404, {"error": "unknown path"})
def do_POST(self):
global next_id
if self.path != "/notes":
return self.send_json(404, {"error": "unknown path"})
length = int(self.headers.get("Content-Length", 0))
body = json.loads(self.rfile.read(length) or b"{}")
if "text" not in body:
return self.send_json(400, {"error": "field 'text' is required"})
with lock:
note = {"id": next_id, "text": body["text"]}
notes[next_id] = note
next_id += 1
self.send_json(201, note)
def log_message(self, *args): # keep the output clean
pass
def call(method, url, payload=None):
data = json.dumps(payload).encode() if payload is not None else None
req = urllib.request.Request(url, data=data, method=method,
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req) as resp:
return resp.status, json.loads(resp.read())
except urllib.error.HTTPError as e:
return e.code, json.loads(e.read())
server = ThreadingHTTPServer(("127.0.0.1", 8000), NotesAPI)
threading.Thread(target=server.serve_forever, daemon=True).start()
base = "http://127.0.0.1:8000"
print("POST /notes ->", *call("POST", base + "/notes", {"text": "Revise OS"}))
print("POST /notes ->", *call("POST", base + "/notes", {"text": "Solve 2 DP problems"}))
print("POST /notes ->", *call("POST", base + "/notes", {"title": "oops"}))
print("GET /notes/1 ->", *call("GET", base + "/notes/1"))
print("GET /notes/9 ->", *call("GET", base + "/notes/9"))
print("GET /notes ->", *call("GET", base + "/notes"))
server.shutdown()
Output:
POST /notes -> 201 {'id': 1, 'text': 'Revise OS'}
POST /notes -> 201 {'id': 2, 'text': 'Solve 2 DP problems'}
POST /notes -> 400 {'error': "field 'text' is required"}
GET /notes/1 -> 200 {'id': 1, 'text': 'Revise OS'}
GET /notes/9 -> 404 {'error': 'note not found'}
GET /notes -> 200 [{'id': 1, 'text': 'Revise OS'}, {'id': 2, 'text': 'Solve 2 DP problems'}]
Things to notice:
The server validates input itself and returns 400; it does not trust the client.
Creating returns 201, a missing note returns 404: the client can react correctly without parsing error text.
The
lockis needed because the threaded server can run two POSTs at the same time; without it, two notes could get the same ID (a race condition from Operating Systems).This server is stateful: notes live in its memory. Restart it and they are gone; run two copies and each has different notes. A real system keeps this state in a database, which is exactly the step from Stage 1 to Stage 2 in the Introduction to System Design chapter.
Good API Habits (Preview)
Use nouns for resources and methods for actions:
GET /notes/7, notGET /getNote?id=7.Paginate lists:
GET /notes?limit=20&cursor=abc. Never return a million rows in one response.Version the API (
/v1/notesor a header) so old mobile apps keep working after changes.Return consistent error bodies with a machine-readable code and a human message.
Set timeouts on every client call. A client that waits forever on a slow server ties up its own threads.
Common Mistakes
Trusting the client. Prices, user IDs and permissions sent by the client must be checked on the server.
Using GET for actions that change data. Crawlers, prefetchers and caches assume GET is safe and may call it on their own.
Returning 200 for every response with an error message inside. Clients, proxies and monitoring rely on correct status codes.
Blindly retrying POST. Without an idempotency key, retries can create duplicates.
Confusing 401 and 403. 401 = "who are you?"; 403 = "I know who you are, and you are not allowed".
Choosing WebSockets for everything. If updates are rare or one-way, polling or SSE is simpler to scale.
Interview Questions
Q1. What does "idempotent" mean? Which HTTP methods are idempotent?
A request is idempotent if sending it many times has the same intended effect on the server as sending it once. According to RFC 9110, GET, HEAD, OPTIONS, TRACE, PUT and DELETE are idempotent. POST is not, and PATCH is not guaranteed to be. Note that the response may differ (a second DELETE may return 404), but the server's state is the same.
Q2. A payment request times out. How do you make it safe for the client to retry?
The client generates a unique idempotency key per payment attempt and sends it with the request. The server stores the key with the result in the same transaction as the payment. If the same key arrives again, the server returns the stored result instead of charging again.
Q3. Why is a 3-tier architecture preferred over clients talking directly to the database?
The middle tier keeps business rules in one place, hides the database (credentials never reach clients), validates input, controls the number of database connections, and lets the app tier and database scale independently.
Q4. When would you choose SSE over WebSockets?
When updates flow only from server to client (live scores, notifications, progress bars). SSE runs over normal HTTP, reconnects automatically in browsers and is simpler to operate. WebSockets are worth the extra complexity when both sides send frequent messages, as in chat or games.
Q5. What is the difference between 502 and 504?
Both come from a gateway or proxy such as a load balancer. 502 Bad Gateway means it got an invalid response (or none, such as a refused or reset connection) from the upstream server. 504 Gateway Timeout means the upstream did not answer within the proxy's time limit.
Summary
In the client-server model, clients send requests and servers own the data and rules. Never trust the client.
A web request goes through DNS, TCP, TLS and HTTP before your code runs; round trips cost time, so connections are reused.
Modern systems are n-tier: presentation, application and data tiers scale separately.
An API is a contract; HTTP methods have defined safety and idempotency, and status codes tell clients what happened.
Idempotency makes retries safe; use idempotency keys for non-idempotent operations like payments.
Keep app servers stateless; push updates with polling, long polling, SSE or WebSockets depending on the need.
The client asks, the server decides, and the API is the promise between them.