Chapter 03 of 33

Client-Server Architecture & APIs

Open any app on your phone and tap a button. Somewhere, a program you cannot see receives a message, does some work and sends back an answer. The app on your phone is the client, the program far away is the server, and the agreed format of their messages is the API.

Almost every system you will design in this tutorial is built from this one idea, repeated many times: browsers talk to web servers, web servers talk to databases, services talk to other services. In this chapter we look at the client-server model from a system designer's point of view: how a request travels, how systems are split into tiers, what an API contract is, how HTTP methods and status codes are used, and how a server can push updates to clients. We finish by building and running a tiny API in Python.

  Client                          Server
 (browser, app)                 (always running)
     │ ─── request: "give me note 7" ───→ │
     │                                    │  does the work
     │ ←── response: 200 OK + the note ── │

The Client-Server Model

In the client-server model, a server is a program that offers a service and waits for requests; a client is a program that sends requests to the server and uses its responses.

A few things follow from this definition:

  • Roles, not machines. "Server" means the program's role. Your laptop can run a server; a big data-centre machine can act as a client of a database.

  • The client starts the conversation. In classic request-response, the server never calls the client out of nowhere. (We will see ways around this later in this chapter.)

  • Centralised data and rules. The server owns the data and enforces the rules. A client can be modified by anyone, so the server must never trust the client: validate every input and check every permission on the server side.

  • One server, many clients. This is why servers must handle many requests at the same time and why scaling servers is the central topic of system design.

The alternative is peer-to-peer (P2P), where every node is both client and server (BitTorrent is the classic example). P2P has no central bottleneck, but it is hard to control, secure and keep consistent. Most products you will design are client-server, often with many servers behind the scenes.

Feature

Client-Server

Peer-to-Peer

Who holds the data

Central servers

Spread across peers

Control and security

Easy to enforce at the server

Hard; peers are untrusted

Scaling

Add server capacity as users grow

Each new peer also adds capacity

Failure

Server down = service down (unless replicated)

No single point of failure

Examples

Websites, banking, email, most apps

BitTorrent, some blockchain networks


The Life of a Web Request

When a user opens https://example.com/notes/7, several steps happen before the server code even runs. You studied these protocols in Computer Networks; here is how they look to a system designer.

1. DNS lookup        example.com → 203.0.113.10 (cached after the first time)
2. TCP connection    3-way handshake with the server  (1 round trip)
3. TLS handshake     agree on encryption keys         (1 more round trip with TLS 1.3)
4. HTTP request      GET /notes/7                     ──→ server
5. Server work       check auth, read cache/database, build the response
6. HTTP response     200 OK + JSON body               ←── server
7. Client renders    the note appears on screen

Why this matters for design:

  • Steps 2 and 3 cost round trips. If the server is far away, each round trip may cost tens or hundreds of milliseconds. That is why connections are reused (HTTP keep-alive, HTTP/2 multiplexing) and why content is moved closer to users with CDNs.

  • Step 5 is where the server's own design matters: how many database queries, whether a cache is hit, whether slow work is pushed to a background queue.

  • With HTTP/3, the transport is QUIC over UDP, which combines the transport and TLS handshakes to save a round trip on new connections.


Tiers: Splitting the System

A tier is a layer of the system that can run on its own machines. Splitting into tiers lets each part scale and fail independently.

1-tier:   [ UI + logic + data ]                       e.g. a desktop app with a local file

2-tier:   [ Client: UI + logic ] ──→ [ Database ]     e.g. old desktop apps talking to a DB

3-tier:   [ Client: UI ] ──→ [ App server: logic ] ──→ [ Database: data ]
          (presentation)      (application)             (data)

Architecture

Pros

Cons

1-tier

Simplest; no network

No sharing between users

2-tier

Shared data; easy to build

Every client connects to the database directly: poor security, hard to change business rules, many DB connections

3-tier / n-tier

Business rules live in one place; database is hidden; each tier scales separately

More network hops and more components to run

Almost all modern web systems are n-tier: the middle tier is often split further into a load balancer, several services, caches and queues. The rest of this tutorial is about how to design that middle and data tier.


What Is an API?

An API (Application Programming Interface) is a contract that defines how one piece of software can request work from another: which operations exist, what inputs they take, what outputs and errors they return.

An API hides the how and exposes only the what. The mobile app does not know whether notes are stored in MySQL or MongoDB; it only knows that GET /notes/7 returns a note. That separation is powerful: the server team can change the database, add a cache or split the service, and as long as the contract stays the same, no client breaks.

In system design, "API" usually means a web API: operations exposed over the network, most often using HTTP and JSON. Popular styles are REST, GraphQL and gRPC; they are compared in detail in the API Design: REST, GraphQL & gRPC chapter. Here we focus on the HTTP basics that all of them build on.

Anatomy of an HTTP request and response

POST /notes HTTP/1.1                     ← method, path, version
Host: api.example.com                    ← headers (metadata)
Content-Type: application/json
Authorization: Bearer eyJhbGciOi...
Content-Length: 21
                                         ← blank line
{"text": "Revise OS"}                    ← body


HTTP/1.1 201 Created                     ← status code and reason
Content-Type: application/json
Location: /notes/1
Content-Length: 30

{"id": 1, "text": "Revise OS"}

HTTP methods

The method tells the server what kind of action the client wants. The current HTTP specification (RFC 9110) defines two important properties:

  • Safe: the method is essentially read-only; the client does not request any change on the server.

  • Idempotent: sending the same request several times has the same intended effect on the server as sending it once.

Method

Typical use

Safe?

Idempotent?

GET

Read a resource

Yes

Yes

HEAD

Like GET but headers only

Yes

Yes

POST

Create something or run an action

No

No

PUT

Create or fully replace a resource at a known URL

No

Yes

PATCH

Partially update a resource (defined in RFC 5789)

No

Not guaranteed

DELETE

Remove a resource

No

Yes

OPTIONS

Ask which methods/options are supported (used by CORS preflight)

Yes

Yes

Why idempotency matters so much in distributed systems: suppose a client sends a request and the network drops the response. The client does not know whether the server did the work, so it retries. Retrying PUT /users/5/name is harmless: the name is just set again. Retrying POST /payments could charge the user twice. That is why payment APIs commonly accept an idempotency key: the client sends a unique ID (for example in an Idempotency-Key header) and the server remembers it, so a repeated request returns the first result instead of charging again. (An Idempotency-Key header is being standardised by the IETF; at the time of writing it is still an Internet-Draft, not an RFC.)

Status codes

The three-digit status code tells the client what happened. The first digit gives the class.

Code

Meaning

When to use it

200 OK

Success

A normal successful GET or action

201 Created

A new resource was created

After POST creates a note; often with a Location header

204 No Content

Success, no body

After a successful DELETE

301 / 308

Moved permanently

Old URL redirects to a new one (308 keeps the method)

304 Not Modified

Your cached copy is still valid

Conditional GET with If-None-Match / ETag

400 Bad Request

The request is malformed

Missing or invalid field

401 Unauthorized

Not authenticated

No valid login token

403 Forbidden

Authenticated but not allowed

A student trying to edit marks

404 Not Found

No such resource

Unknown note ID

409 Conflict

Request conflicts with current state

Username already taken; version mismatch

429 Too Many Requests

Client is sending too fast (RFC 6585)

Rate limit exceeded; see the Rate Limiting chapter

500 Internal Server Error

Bug or unexpected failure on the server

Unhandled exception

502 Bad Gateway

A proxy got a bad response from the upstream server

Load balancer cannot get a valid reply from an app server

503 Service Unavailable

Temporarily overloaded or down

Maintenance or overload; may include Retry-After

504 Gateway Timeout

A proxy timed out waiting for the upstream

App server too slow to answer the load balancer

Rule of thumb: 4xx means "the client should change something"; retrying the same request will not help (except 429, after waiting). 5xx means "the server had a problem"; a retry later may succeed, but only retry idempotent requests or requests protected by an idempotency key.


Stateless vs Stateful Servers

HTTP itself is stateless: each request carries everything needed to understand it, and the protocol does not remember earlier requests. Applications still need state ("Asha is logged in", "her cart has 2 items"). The question is where that state lives.

Stateful server

Stateless server

Keeps session data in its own memory

Keeps no per-user data between requests

The next request must reach the same server

Any server can handle any request

If the server dies, its sessions are lost

A server can die or be added at any time

Harder to scale out

Easy to scale out behind a load balancer

To make app servers stateless, the state moves elsewhere: into a token the client sends with every request (such as a signed JWT, covered in Authentication & Authorization: OAuth & JWT), or into a shared store such as Redis or the database. The next chapter, Scalability: Vertical vs Horizontal, shows with running code why this matters.


How Servers Send Updates to Clients

Classic request-response has a limitation: the server can only answer, never start a conversation. But chat apps, live scores and stock tickers need the server to tell the client "something changed". There are four common patterns.

Short polling:  C: any news? → S: no   C: any news? → S: no   C: any news? → S: yes!
Long polling:   C: any news? → S: (holds the request open ... ) → S: yes! → C: any news? ...
SSE:            C: subscribe → S: event → S: event → S: event ...     (one-way, server→client)
WebSocket:      C ⇄ S  both sides send messages at any time over one connection

Pattern

How it works

Good for

Drawbacks

Short polling

Client asks every few seconds

Simple dashboards; works with plain HTTP caches and CDNs

Many empty responses; updates delayed up to one interval

Long polling

Server holds the request until there is news or a timeout, then the client immediately asks again

Near-real-time where WebSockets are blocked

Server keeps many waiting requests; reconnect after every message

Server-Sent Events (SSE)

One long HTTP response streams events (text/event-stream); browsers use the EventSource API, which reconnects automatically

Live scores, notifications, streaming text: server-to-client only

One direction only; client sends via normal requests

WebSocket (RFC 6455)

Starts as an HTTP request, then upgrades to a persistent, full-duplex connection

Chat, multiplayer games, collaborative editing

Long-lived connections are stateful: harder to load balance and scale

We will use these patterns in the case studies, especially Design a Chat Application.


Hands-On: A Tiny Notes API in Python

Let us build a real client and server using only Python's standard library. The server keeps notes in memory and supports POST /notes, GET /notes and GET /notes/{id}. The same script then acts as a client and calls the server, so you can see the status codes from the tables above in action.

import json
import threading
import urllib.request
import urllib.error
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

notes = {}          # the server's state: id -> note
next_id = 1
lock = threading.Lock()   # ThreadingHTTPServer handles requests in parallel

class NotesAPI(BaseHTTPRequestHandler):
    def send_json(self, status, body):
        data = json.dumps(body).encode()
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        self.wfile.write(data)

    def do_GET(self):
        if self.path == "/notes":
            self.send_json(200, list(notes.values()))
        elif self.path.startswith("/notes/"):
            note = notes.get(int(self.path.split("/")[-1]))
            if note:
                self.send_json(200, note)
            else:
                self.send_json(404, {"error": "note not found"})
        else:
            self.send_json(404, {"error": "unknown path"})

    def do_POST(self):
        global next_id
        if self.path != "/notes":
            return self.send_json(404, {"error": "unknown path"})
        length = int(self.headers.get("Content-Length", 0))
        body = json.loads(self.rfile.read(length) or b"{}")
        if "text" not in body:
            return self.send_json(400, {"error": "field 'text' is required"})
        with lock:
            note = {"id": next_id, "text": body["text"]}
            notes[next_id] = note
            next_id += 1
        self.send_json(201, note)

    def log_message(self, *args):   # keep the output clean
        pass

def call(method, url, payload=None):
    data = json.dumps(payload).encode() if payload is not None else None
    req = urllib.request.Request(url, data=data, method=method,
                                 headers={"Content-Type": "application/json"})
    try:
        with urllib.request.urlopen(req) as resp:
            return resp.status, json.loads(resp.read())
    except urllib.error.HTTPError as e:
        return e.code, json.loads(e.read())

server = ThreadingHTTPServer(("127.0.0.1", 8000), NotesAPI)
threading.Thread(target=server.serve_forever, daemon=True).start()

base = "http://127.0.0.1:8000"
print("POST /notes    ->", *call("POST", base + "/notes", {"text": "Revise OS"}))
print("POST /notes    ->", *call("POST", base + "/notes", {"text": "Solve 2 DP problems"}))
print("POST /notes    ->", *call("POST", base + "/notes", {"title": "oops"}))
print("GET  /notes/1  ->", *call("GET", base + "/notes/1"))
print("GET  /notes/9  ->", *call("GET", base + "/notes/9"))
print("GET  /notes    ->", *call("GET", base + "/notes"))
server.shutdown()

Output:

POST /notes    -> 201 {'id': 1, 'text': 'Revise OS'}
POST /notes    -> 201 {'id': 2, 'text': 'Solve 2 DP problems'}
POST /notes    -> 400 {'error': "field 'text' is required"}
GET  /notes/1  -> 200 {'id': 1, 'text': 'Revise OS'}
GET  /notes/9  -> 404 {'error': 'note not found'}
GET  /notes    -> 200 [{'id': 1, 'text': 'Revise OS'}, {'id': 2, 'text': 'Solve 2 DP problems'}]

Things to notice:

  • The server validates input itself and returns 400; it does not trust the client.

  • Creating returns 201, a missing note returns 404: the client can react correctly without parsing error text.

  • The lock is needed because the threaded server can run two POSTs at the same time; without it, two notes could get the same ID (a race condition from Operating Systems).

  • This server is stateful: notes live in its memory. Restart it and they are gone; run two copies and each has different notes. A real system keeps this state in a database, which is exactly the step from Stage 1 to Stage 2 in the Introduction to System Design chapter.


Good API Habits (Preview)

  • Use nouns for resources and methods for actions: GET /notes/7, not GET /getNote?id=7.

  • Paginate lists: GET /notes?limit=20&cursor=abc. Never return a million rows in one response.

  • Version the API (/v1/notes or a header) so old mobile apps keep working after changes.

  • Return consistent error bodies with a machine-readable code and a human message.

  • Set timeouts on every client call. A client that waits forever on a slow server ties up its own threads.


Common Mistakes

  • Trusting the client. Prices, user IDs and permissions sent by the client must be checked on the server.

  • Using GET for actions that change data. Crawlers, prefetchers and caches assume GET is safe and may call it on their own.

  • Returning 200 for every response with an error message inside. Clients, proxies and monitoring rely on correct status codes.

  • Blindly retrying POST. Without an idempotency key, retries can create duplicates.

  • Confusing 401 and 403. 401 = "who are you?"; 403 = "I know who you are, and you are not allowed".

  • Choosing WebSockets for everything. If updates are rare or one-way, polling or SSE is simpler to scale.


Interview Questions

Q1. What does "idempotent" mean? Which HTTP methods are idempotent?

A request is idempotent if sending it many times has the same intended effect on the server as sending it once. According to RFC 9110, GET, HEAD, OPTIONS, TRACE, PUT and DELETE are idempotent. POST is not, and PATCH is not guaranteed to be. Note that the response may differ (a second DELETE may return 404), but the server's state is the same.

Q2. A payment request times out. How do you make it safe for the client to retry?

The client generates a unique idempotency key per payment attempt and sends it with the request. The server stores the key with the result in the same transaction as the payment. If the same key arrives again, the server returns the stored result instead of charging again.

Q3. Why is a 3-tier architecture preferred over clients talking directly to the database?

The middle tier keeps business rules in one place, hides the database (credentials never reach clients), validates input, controls the number of database connections, and lets the app tier and database scale independently.

Q4. When would you choose SSE over WebSockets?

When updates flow only from server to client (live scores, notifications, progress bars). SSE runs over normal HTTP, reconnects automatically in browsers and is simpler to operate. WebSockets are worth the extra complexity when both sides send frequent messages, as in chat or games.

Q5. What is the difference between 502 and 504?

Both come from a gateway or proxy such as a load balancer. 502 Bad Gateway means it got an invalid response (or none, such as a refused or reset connection) from the upstream server. 504 Gateway Timeout means the upstream did not answer within the proxy's time limit.


Summary

  • In the client-server model, clients send requests and servers own the data and rules. Never trust the client.

  • A web request goes through DNS, TCP, TLS and HTTP before your code runs; round trips cost time, so connections are reused.

  • Modern systems are n-tier: presentation, application and data tiers scale separately.

  • An API is a contract; HTTP methods have defined safety and idempotency, and status codes tell clients what happened.

  • Idempotency makes retries safe; use idempotency keys for non-idempotent operations like payments.

  • Keep app servers stateless; push updates with polling, long polling, SSE or WebSockets depending on the need.

The client asks, the server decides, and the API is the promise between them.