When you type cat access.log | grep error | wc -l in a terminal, three separate programs run as three separate processes, yet data flows smoothly from the first to the second to the third. How do processes, which have completely separate memory, pass data to each other?
In Process Creation (fork & exec) we saw that a child's changes to a variable are invisible to its parent. Each process lives in its own protected address space. That protection is great for safety, but processes often need to cooperate. The OS therefore provides controlled ways to cross the wall between processes.
Inter-Process Communication (IPC) is the set of mechanisms an operating system provides so that processes can exchange data and coordinate their actions.
Independent vs Cooperating Processes
An independent process cannot affect or be affected by other processes. It shares no data with anyone.
A cooperating process can affect or be affected by other processes, for example by sharing data.
Why build systems out of cooperating processes?
Information sharing: several programs need the same data (e.g. copy-paste between applications).
Computation speedup: split a task into parts that run in parallel on multiple cores.
Modularity: build a system from small separate programs, each doing one job (the Unix philosophy).
Isolation with cooperation: modern browsers run each site in its own process for security, but those processes must still talk to the main browser process.
The Two Fundamental Models
All IPC mechanisms fall into one of two models.
SHARED MEMORY MESSAGE PASSING
┌───────────┐ ┌───────────┐
│ Process A │──┐ │ Process A │── send(msg) ──┐
├───────────┤ │ └───────────┘ │
│ shared │◀─┼─ both read/write ▼
│ region │ │ directly ┌───────────────────────────────┐
├───────────┤ │ │ Kernel (message queue) │
│ Process B │──┘ └───────────────────────────────┘
└───────────┘ │
┌───────────┐ │
Kernel is involved only in │ Process B │◀─ receive() ──┘
SETTING UP the region └───────────┘
Kernel is involved in EVERY message
Shared Memory | Message Passing |
|---|---|
Processes read and write a common region of memory | Processes exchange messages via |
Kernel needed only to create/attach the region | Kernel (usually) copies every message |
Fastest: data is accessed at memory speed | Slower for large data: system call + copy per message |
Programmer must synchronize access (avoid races) | Synchronization is largely built in (receive waits for a message) |
Works only on the same machine | Easy to extend across machines (sockets) |
Examples: POSIX | Examples: pipes, FIFOs, message queues, sockets |
The rule of thumb: shared memory for large amounts of data and speed; message passing for simplicity and small messages.
Design Choices in Message Passing
Every message-passing system answers three questions.
1. Naming: Direct or Indirect?
Direct communication: processes name each other explicitly:
send(P, msg),receive(Q, msg). A link exists between exactly one pair.Indirect communication: messages are sent to and received from a mailbox (also called a port or queue):
send(M, msg),receive(M, msg). Several processes can share one mailbox, and senders and receivers do not need to know each other. POSIX message queues are mailboxes.
2. Synchronization: Blocking or Non-Blocking?
Operation | Blocking (synchronous) | Non-blocking (asynchronous) |
|---|---|---|
send | Sender waits until the message is received (or accepted into the buffer) | Sender hands over the message and continues immediately |
receive | Receiver waits until a message is available | Receiver gets a message, or "nothing available" immediately |
When both send and receive are blocking, sender and receiver meet at the same moment; this is called a rendezvous.
3. Buffering: How Many Messages Can Wait?
Zero capacity: no messages can wait. The sender must block until the receiver takes the message (rendezvous).
Bounded capacity: up to n messages can wait. If the queue is full, the sender blocks. This is what real systems use.
Unbounded capacity: the sender never blocks. This is a theoretical model, since real memory is finite.
Pipes
A pipe is the oldest and simplest Unix IPC mechanism: a one-way channel of bytes managed by the kernel. Data written into one end comes out of the other end in the same order (first in, first out).
write() read()
Process A ──────────▶ fd[1] ═══ pipe ═══ fd[0] ──────────▶ Process B
(write end) kernel (read end)
buffer
The pipe(fd) system call creates a pipe and returns two file descriptors: fd[0] for reading and fd[1] for writing. A pipe has no name, so only related processes can use it: the usual pattern is to create the pipe and then fork(), because the child inherits copies of both descriptors.
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <unistd.h>
int main(void) {
int fd[2]; /* fd[0] = read end, fd[1] = write end */
if (pipe(fd) == -1) { perror("pipe"); return 1; }
pid_t pid = fork();
if (pid == 0) { /* CHILD: reader */
close(fd[1]); /* child does not write */
char buf[100];
ssize_t n = read(fd[0], buf, sizeof buf - 1);
buf[n > 0 ? n : 0] = '\0';
printf("Child received: \"%s\" (%zd bytes)\n", buf, n);
close(fd[0]);
return 0;
}
/* PARENT: writer */
close(fd[0]); /* parent does not read */
const char *msg = "Hello child, this is your parent";
write(fd[1], msg, strlen(msg));
close(fd[1]); /* reader will now see end-of-file */
wait(NULL);
return 0;
}
Compile and run:
gcc -std=c17 -Wall -Wextra pipe1.c -o pipe1
./pipe1
Output:
Child received: "Hello child, this is your parent" (32 bytes)
Each process closes the end it does not use. This is not just tidiness. The reader sees end-of-file (read() returns 0) only when every write end is closed. If the child forgot to close its own copy of fd[1], it could wait forever for more data that will never come.
How Pipes Behave
Unidirectional: data flows one way. For two-way communication, use two pipes.
Byte stream: no message boundaries. Two writes of 10 bytes may be read as one read of 20 bytes.
Bounded buffer: on Linux the default capacity is 64 KiB. Checking with
fcntl(fd, F_GETPIPE_SZ)on our test machine printed:
Pipe capacity: 65536 bytes
Blocking:
read()on an empty pipe blocks until data arrives;write()to a full pipe blocks until the reader makes space. So a pipe automatically slows a fast writer down to the reader's speed.Broken pipe: writing to a pipe whose read ends are all closed sends the writer the
SIGPIPEsignal (which kills it by default) or, if the signal is ignored, makeswrite()fail withEPIPE.
How the Shell Builds ls | wc -l
A shell pipeline combines everything from the previous chapter: pipe(), fork(), and exec(), plus dup2(), which makes one file descriptor a copy of another. Here is a program that does exactly what the shell does for ls / | wc -l:
#include <stdio.h>
#include <sys/wait.h>
#include <unistd.h>
/* Do what the shell does for: ls / | wc -l */
int main(void) {
int fd[2];
pipe(fd);
if (fork() == 0) { /* first child: ls / */
dup2(fd[1], STDOUT_FILENO); /* stdout -> pipe write end */
close(fd[0]); close(fd[1]);
execlp("ls", "ls", "/", (char *)NULL);
_exit(127);
}
if (fork() == 0) { /* second child: wc -l */
dup2(fd[0], STDIN_FILENO); /* stdin <- pipe read end */
close(fd[0]); close(fd[1]);
execlp("wc", "wc", "-l", (char *)NULL);
_exit(127);
}
close(fd[0]); close(fd[1]); /* parent MUST close both ends */
wait(NULL);
wait(NULL);
return 0;
}
Output:
22
This matches running ls / | wc -l in the shell on the same machine (22 entries in /; your number will differ). Neither ls nor wc knows anything about pipes: ls just writes to its stdout and wc reads its stdin. The shell rewired those descriptors between fork and exec, which is exactly why Unix separates the two calls.
Named Pipes (FIFOs)
An ordinary pipe disappears when the processes using it close it, and unrelated processes cannot find it. A named pipe, or FIFO, has a name in the file system, so any process with permission can open it.
mkfifo myfifo
ls -l myfifo
echo 'hello through a FIFO' > myfifo & # writer (blocks until a reader opens)
cat myfifo # reader
Output:
prw-r--r-- 1 imran imran 0 Nov 2 08:17 myfifo
hello through a FIFO
The p at the start of the permissions means "pipe". Its size is 0: the data never goes to disk; the kernel passes it through memory exactly like an ordinary pipe. Opening a FIFO for writing blocks until some process opens it for reading (and vice versa). In C, a FIFO is created with mkfifo(path, mode) and then used with ordinary open, read and write.
Message Queues
A message queue is a kernel-managed mailbox that stores whole messages, not a stream of bytes. Each receive gets exactly one complete message. POSIX message queues also give each message a priority: higher-priority messages are delivered first.
#include <fcntl.h>
#include <mqueue.h>
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <unistd.h>
#define QNAME "/demo_queue"
int main(void) {
struct mq_attr attr = { .mq_maxmsg = 10, .mq_msgsize = 64 };
mqd_t q = mq_open(QNAME, O_CREAT | O_RDWR, 0600, &attr);
if (q == (mqd_t)-1) { perror("mq_open"); return 1; }
if (fork() == 0) { /* child: sender */
const char *msgs[] = {"order #1", "order #2", "URGENT: refund"};
unsigned prio[] = {1, 1, 5};
for (int i = 0; i < 3; i++)
mq_send(q, msgs[i], strlen(msgs[i]) + 1, prio[i]);
return 0;
}
wait(NULL); /* let the sender finish first */
char buf[64];
unsigned p;
for (int i = 0; i < 3; i++) { /* parent: receiver */
mq_receive(q, buf, sizeof buf, &p);
printf("Received (priority %u): %s\n", p, buf);
}
mq_close(q);
mq_unlink(QNAME);
return 0;
}
Output:
Received (priority 5): URGENT: refund
Received (priority 1): order #1
Received (priority 1): order #2
The "URGENT" message was sent last, but was received first because of its higher priority (5). Messages with equal priority keep their sending order. The queue was created with mq_maxmsg = 10: it is a bounded buffer, so an 11th unread message would make mq_send() block. (On older glibc versions you must link with -lrt; since glibc 2.34 these functions are in the main C library.)
Unix also has an older API, System V message queues (msgget, msgsnd, msgrcv), which you may see in older textbooks and exam questions. The idea is the same.
Shared Memory
With shared memory, the kernel maps the same physical memory into the address spaces of two or more processes. After setup, the processes read and write it like ordinary memory, with no system calls at all. This makes it the fastest IPC method.
The POSIX steps are:
shm_open()creates (or opens) a named shared memory object.ftruncate()sets its size.mmap()maps it into the process's address space withMAP_SHARED.Use it through the returned pointer.
munmap(),close()and finallyshm_unlink()to remove it.
#define _POSIX_C_SOURCE 200809L /* expose POSIX functions under -std=c17 */
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>
#define NAME "/demo_shm"
#define SIZE 4096
int main(void) {
/* 1. Create a named shared memory object and give it a size */
int fd = shm_open(NAME, O_CREAT | O_RDWR, 0600);
if (fd == -1) { perror("shm_open"); return 1; }
ftruncate(fd, SIZE);
/* 2. Map it into our address space */
char *mem = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (mem == MAP_FAILED) { perror("mmap"); return 1; }
pid_t pid = fork();
if (pid == 0) { /* child: writer */
strcpy(mem, "Data written by the child directly into shared memory");
return 0;
}
wait(NULL); /* crude synchronization: wait */
printf("Parent read: %s\n", mem); /* parent: reader */
/* 3. Clean up */
munmap(mem, SIZE);
close(fd);
shm_unlink(NAME);
return 0;
}
Output:
Parent read: Data written by the child directly into shared memory
Compare this with the fork example where a child's change to x was invisible: here the child's write is visible to the parent, because the region was mapped with MAP_SHARED, so copy-on-write does not apply to it. On Linux, POSIX shared memory objects appear as files under /dev/shm.
The Catch: Synchronization
Our example "synchronized" in the crudest possible way: the parent waited for the child to exit before reading. Real programs run reader and writer at the same time, and then serious problems appear:
The reader may read while the writer is halfway through, getting half-old, half-new data.
Two writers may update the same value simultaneously and lose an update.
The reader has no built-in way to know that new data has arrived.
Shared memory gives you speed, but you must add synchronization yourself, typically with semaphores or mutexes (see Mutex Locks & Semaphores). The classic pattern is the producer-consumer problem: a producer puts items into a shared, fixed-size buffer and a consumer takes them out. The producer must wait when the buffer is full, and the consumer must wait when it is empty. A pipe solves this automatically inside the kernel; with shared memory you must build it yourself, as shown in Classical Synchronization Problems.
Signals
A signal is a small asynchronous notification sent to a process: it carries only a number, not data. Signals tell a process that an event happened: SIGINT (Ctrl+C), SIGTERM (please terminate), SIGKILL (terminate now, cannot be caught), SIGCHLD (a child has stopped or exited), SIGSEGV (invalid memory access), and two free for applications to use, SIGUSR1 and SIGUSR2.
A process can ignore most signals, accept the default action (often termination), or install a handler function. Signals are sent with the kill() system call or the kill command.
#define _POSIX_C_SOURCE 200809L /* expose POSIX functions under -std=c17 */
#include <signal.h>
#include <stdio.h>
#include <sys/wait.h>
#include <unistd.h>
volatile sig_atomic_t got = 0;
void handler(int sig) { got = sig; }
int main(void) {
struct sigaction sa = {0};
sa.sa_handler = handler;
sigaction(SIGUSR1, &sa, NULL); /* install handler BEFORE fork */
pid_t pid = fork();
if (pid == 0) { /* child: wait for a signal */
while (!got)
pause(); /* sleep until a signal arrives */
printf("Child : got signal %d (SIGUSR1)\n", got);
return 0;
}
sleep(1);
printf("Parent: sending SIGUSR1 to child %d\n", pid);
fflush(stdout);
kill(pid, SIGUSR1);
wait(NULL);
return 0;
}
Output:
Parent: sending SIGUSR1 to child 19230
Child : got signal 10 (SIGUSR1)
(SIGUSR1 is signal number 10 on Linux for x86 and ARM; some other systems use a different number, which is why code should always use the name SIGUSR1, never the raw number.)
Signals are good for notifications ("reload your config", "your child exited"), but not for transferring data. This example is deliberately simple: in production code, checking got and then calling pause() has a tiny window in which a signal could arrive in between and be missed; robust code uses sigsuspend() with a blocked signal mask instead.
Sockets
A socket is an endpoint for two-way communication. Sockets are the only common IPC mechanism that works both on the same machine and across a network:
Unix domain sockets (
AF_UNIX) connect processes on the same machine through a path in the file system. They are bidirectional and fast. For example, Docker's command-line tool talks to the Docker daemon through/var/run/docker.sock, and MySQL clients often connect through a local socket file.Network sockets (TCP/UDP over IP) connect processes on different machines. A web browser and a web server are two processes communicating through sockets.
Socket programming is covered in detail in the Computer Networks tutorial. From an OS point of view, sockets are simply message passing with a very flexible naming scheme (IP address + port, or a file path).
Other IPC Mechanisms You Will Hear About
Memory-mapped files: two processes
mmap()the same file withMAP_SHARED, and changes by one are seen by the other (and eventually written to the file).Remote Procedure Call (RPC): lets a process call a function in another process (possibly on another machine) as if it were local; modern examples include gRPC.
Binder: Android's main IPC mechanism, used by apps to talk to system services.
D-Bus: a message bus used by Linux desktop applications and system services.
Comparing IPC Mechanisms
Mechanism | Model | Direction | Related processes only? | Data form | Typical use |
|---|---|---|---|---|---|
Pipe | Message passing | One-way | Yes (inherited via fork) | Byte stream | Shell pipelines, parent ↔ child |
FIFO (named pipe) | Message passing | One-way | No (has a file name) | Byte stream | Simple unrelated producer/consumer |
Message queue | Message passing | Any (mailbox) | No | Discrete messages, with priority | Task queues, commands |
Shared memory | Shared memory | Any | No | Raw memory | Large data, high speed (needs sync) |
Signal | Notification | One-way | No | Just a signal number | Events: stop, reload, child exited |
Socket | Message passing | Two-way | No; even across machines | Stream or datagrams | Client-server, networking |
Common Mistakes
Not closing unused pipe ends. The reader then never sees end-of-file and hangs forever.
Expecting message boundaries in a pipe. A pipe is a byte stream. If you need messages, add a length prefix or a delimiter, or use a message queue.
Thinking a pipe is two-way. A classic pipe is one-way; create two pipes for a conversation.
Using shared memory without synchronization. It "works" in small tests and fails randomly under load.
Forgetting to unlink. Named objects (FIFOs, POSIX message queues, shared memory) stay in the system after the program exits until they are removed with
unlink,mq_unlinkorshm_unlink.Using signals to send data. A signal carries only its number, and multiple instances of the same standard signal can merge into one.
Interview Questions
What is IPC and why is it needed?
Inter-process communication is the set of OS mechanisms that let processes exchange data and coordinate. It is needed because each process has its own protected address space, so processes cannot simply read each other's variables.
Compare shared memory and message passing.
Shared memory maps a common region into both processes; it is the fastest because the kernel is involved only at setup, but the programmer must synchronize access. Message passing sends data through the kernel with send/receive; it is simpler and works across machines, but each message costs a system call and a copy.
What is the difference between a pipe and a named pipe (FIFO)?
An ordinary pipe has no name and exists only while open, so it can be used only by related processes that inherit its descriptors. A FIFO has a name in the file system, so unrelated processes can open it.
What happens when a process reads from an empty pipe? When it writes to a full pipe?
By default, read blocks until data is available (or returns 0 for end-of-file if all write ends are closed). Write blocks until the reader frees space. If no reader exists at all, the writer gets SIGPIPE / EPIPE.
Which IPC mechanism is the fastest and why?
Shared memory, because after setup the processes read and write memory directly without system calls or copying data through the kernel.
What is the difference between blocking and non-blocking send/receive?
A blocking send waits until the message is accepted; a blocking receive waits until a message is available. Non-blocking versions return immediately. Blocking send plus blocking receive is called a rendezvous.
How does the shell implement cmd1 | cmd2?
It calls pipe(), then forks two children. In the first it uses dup2 to make stdout the pipe's write end and execs cmd1; in the second it makes stdin the pipe's read end and execs cmd2. The parent closes both pipe ends and waits for both children.
Summary
Processes have separate address spaces, so they need IPC to cooperate.
Two models: shared memory (fast, needs synchronization) and message passing (simpler, kernel-mediated).
Message passing design choices: direct vs indirect naming, blocking vs non-blocking, and zero / bounded / unbounded buffering.
Pipes: one-way byte streams between related processes; the basis of shell pipelines. FIFOs: pipes with a file-system name.
Message queues: mailboxes of discrete messages, with priorities in POSIX.
Shared memory:
shm_open+mmap; fastest, but you must synchronize.Signals notify; sockets communicate both locally and across networks.
Message passing: "let the kernel carry it." Shared memory: "let's both look at the same whiteboard, and take turns writing."