Early electronic computers were "programmed" by plugging cables and setting switches. To make ENIAC solve a different problem, operators could spend days rewiring it. Imagine having to open your laptop and move wires every time you wanted to switch from a browser to a music player.
The idea that fixed this is so simple that we now take it for granted: store the program in memory, just like data. Then changing the program means loading different bytes, not rewiring the machine. In this chapter we study this stored-program concept, the Von Neumann architecture built on it, its famous bottleneck, the alternative Harvard architecture with separate instruction and data memories, and the modified Harvard design that almost every modern processor actually uses.
The Stored-Program Concept
Stored-program concept: a program's instructions are encoded as binary numbers and kept in the computer's memory, from where the processor fetches and executes them one after another. Instructions and data are both just bit patterns in memory.
This idea was described in John von Neumann's 1945 report First Draft of a Report on the EDVAC, drawing on work with J. Presper Eckert and John Mauchly, the builders of ENIAC. The first machine to actually run a stored program was the Manchester "Baby" (1948), followed by practical machines such as EDSAC (1949).
Why was it such a big deal?
Flexibility: load a new program and the same hardware does a completely different job.
Programs that handle programs: because code is just data, one program can read, create or translate another. Compilers, assemblers, loaders and operating systems all depend on this.
Speed: instructions come from fast electronic memory instead of slow mechanical input such as paper tape.
Von Neumann Architecture
A Von Neumann architecture (also called Princeton architecture) has a single memory that holds both instructions and data, connected to the CPU by a single path (one set of address and data buses).
+-------------------------------------+
| CPU |
| +--------------+ +-------------+ |
| | Control Unit | | ALU | |
| +--------------+ +-------------+ |
| PC IR MAR MDR registers |
+-----------------+-------------------+
|
one address bus + one data bus
(used for instruction fetch AND data access)
|
+-----------------+-------------------+
| Single main memory |
| [ instructions ] [ data ] |
+-------------------------------------+
|
I/O
Key Features
One memory for instructions and data, with a single address space.
Instructions are fetched from memory and executed sequentially; the Program Counter (PC) holds the address of the next instruction, unless a branch changes it.
The CPU consists of an ALU, a control unit and registers.
The memory cannot tell instructions from data; the meaning of a word depends only on how the CPU uses it.
The IAS Machine
The design from the EDVAC report was built at Princeton's Institute for Advanced Study as the IAS computer (completed in 1952). It is a useful concrete picture of an early Von Neumann machine:
Memory of 1024 words, each 40 bits long (some textbooks quote 1000 or 4096 words; the 12-bit address field could name up to 4096). A word held either one number or two 20-bit instructions.
Each instruction had an 8-bit opcode and a 12-bit address.
Registers: AC (accumulator) and MQ (multiplier-quotient) for arithmetic; PC, IR, MAR and MBR (memory buffer register, the same idea as MDR); and an IBR (instruction buffer register) to hold the second instruction of a fetched word.
Notice the IBR: fetching two instructions per memory access was an early trick to reduce memory traffic, which brings us to the main weakness of the design.
The Von Neumann Bottleneck
Every instruction must be fetched from memory, and many instructions also read or write data in memory. In a Von Neumann machine, both kinds of traffic share one path, so they cannot happen at the same time.
Von Neumann bottleneck: the limit on performance caused by the single, shared path between the CPU and memory. The processor often sits idle waiting for instructions or data to travel over it. (The term was popularised by John Backus in his 1977 Turing Award lecture.)
The problem has become worse over time, because processor speeds grew much faster than memory speeds. A modern core can execute several instructions per nanosecond, while a trip to main memory takes tens of nanoseconds. Without help, the CPU would spend most of its time waiting.
Ways to reduce the bottleneck (each has its own chapter later):
Caches: keep frequently used instructions and data in small fast memories close to the CPU.
Separate instruction and data paths near the CPU (the Harvard idea, below).
Wider buses and prefetching: bring more bytes per access, and fetch instructions before they are needed.
Harvard Architecture
A Harvard architecture uses physically separate memories for instructions and data, each with its own buses. The name comes from the Harvard Mark I (1944), an electromechanical computer that read its instructions from punched paper tape while keeping data in separate counters.
+--------------------+ +--------------------+
| Instruction memory | | Data memory |
| (program only) | | (data only) |
+---------+----------+ +----------+---------+
| |
instruction address bus data address bus
+ instruction bus + data bus (read/write)
| |
| +-------------------------+ |
+----------->| CPU |<-------+
| Control Unit, ALU, |
| PC, IR, registers |
+-------------------------+
Key Features
Two separate memories and two separate sets of buses.
The CPU can fetch the next instruction and read or write data in the same clock cycle, since the two use different paths.
The two memories have separate address spaces: address 100 in instruction memory and address 100 in data memory are different locations.
Instruction and data words can have different widths. For example, many PIC microcontrollers use 12-, 14- or 16-bit instruction words with 8-bit data.
Where Is Pure Harvard Used?
Microcontrollers such as AVR (used in Arduino Uno), PIC and the 8051 family: the program sits in flash, the data in SRAM. Reading a constant table stored in flash needs special instructions (
LPMon AVR,MOVCon 8051), precisely because program memory is a different address space.Digital Signal Processors (DSPs), which must fetch an instruction and several operands every cycle for filters and FFTs. Analog Devices even named a DSP family SHARC, short for "Super Harvard Architecture".
Von Neumann vs Harvard: Comparison
Feature | Von Neumann | Harvard |
|---|---|---|
Memory for instructions and data | One shared memory | Two separate memories |
Buses | One set shared by both | Separate sets for instructions and data |
Address space | Single | Two independent ones |
Fetch and data access in the same cycle? | No, they take turns | Yes, in parallel |
Speed | Limited by the Von Neumann bottleneck | Higher memory bandwidth |
Hardware cost and complexity | Lower: one memory, one bus | Higher: two memories, more pins and wires |
Memory usage | Flexible: any free space can hold code or data | Fixed split: spare data memory cannot be used for code, or vice versa |
Instruction and data word widths | Same | Can differ |
Self-modifying code, loading programs as data | Easy (code is data) | Hard or impossible without special paths |
Typical use | General-purpose computers (at the main-memory level) | Microcontrollers, DSPs |
Modified Harvard: What Real Processors Use
Your laptop runs programs that are loaded from disk as data and then executed, and a JavaScript engine generates machine code at run time. That needs a single memory where code is data, so main memory is Von Neumann. But the CPU also wants to fetch instructions and data in parallel every cycle, which needs Harvard. The solution combines both:
Modified Harvard architecture: one unified main memory and address space (Von Neumann), but separate paths for instructions and data close to the CPU, usually split level-1 caches: an instruction cache (L1-I) and a data cache (L1-D).
+-------------------------------+
| CPU |
| fetch unit load/store |
+------+-----------------+------+
| |
+-----+-----+ +-----+-----+
| L1-I | | L1-D | <- Harvard-like: parallel access
+-----+-----+ +-----+-----+
| |
+-----+-----------------+-----+
| Unified L2 / L3 cache | <- Von Neumann-like: shared
+--------------+--------------+
|
+--------------+--------------+
| Main memory (code + data) |
+-----------------------------+
When the needed instructions and data are in the L1 caches (which is most of the time), the processor enjoys Harvard-style parallel access. On a miss, both go to the same shared memory, keeping the Von Neumann flexibility.
Examples:
Modern x86, ARM application processors and RISC-V cores have separate L1 instruction and data caches with unified lower levels.
ARM Cortex-M3 and M4 microcontrollers use separate instruction (I-Code) and data (D-Code) buses plus a system bus, all within one unified address map. The smaller Cortex-M0, in contrast, has a single bus for both, a Von Neumann design chosen to save area and power.
One consequence: if a program writes new instructions (as a JIT compiler does), the new bytes go through the data side and may not yet be visible to the instruction side. x86 hardware keeps the two consistent automatically, while on ARM the software must explicitly clean and invalidate the caches (in GCC, __builtin___clear_cache) before executing the new code.
Code as Data: A Security Angle
Because a Von Neumann memory cannot tell code from data, an attacker who overflows a buffer can write machine code into memory and trick the program into jumping to it. Modern systems add back a Harvard-like separation through page permissions: the No-eXecute (NX) bit marks data pages as non-executable, and operating systems follow a "write XOR execute" policy so that a page is never writable and executable at the same time. It is the same lesson as the architecture itself: keeping instructions and data apart has real benefits.
Solved Problems
Problem 1: Memory time with one memory vs two
A program executes 1000 instructions. 30% of them are loads or stores, each needing one data access. Every memory access takes 10 ns. Ignoring everything except memory access time, compare a Von Neumann machine (one memory) with a Harvard machine (separate memories that work in parallel).
Instruction fetches = 1000
Data accesses = 0.30 x 1000 = 300
Von Neumann: all accesses share one memory, one after another
total = (1000 + 300) x 10 ns = 13,000 ns = 13 us
Harvard: the two memories work in parallel
instruction memory busy = 1000 x 10 ns = 10 us
data memory busy = 300 x 10 ns = 3 us
with perfect overlap, time is set by the busier memory = 10 us
Speedup = 13 / 10 = 1.3
Problem 2: Structural hazard in a pipeline
A 5-stage pipeline ideally completes one instruction per cycle (CPI = 1). Instruction fetch happens in stage 1, and loads/stores access memory in stage 4. With a single-ported unified memory, whenever a load or store is in stage 4, the instruction fetch in stage 1 must wait one cycle. 30% of instructions are loads or stores. Find the effective CPI and the cycles for 1000 instructions, with and without split instruction/data memories.
Unified memory (Von Neumann):
stall cycles per instruction = 0.30 x 1 = 0.30
effective CPI = 1 + 0.30 = 1.30
cycles for 1000 instructions = (5 - 1) + 1000 + 300 = 1304
Split memories / split L1 caches (Harvard):
no conflict, CPI = 1
cycles = (5 - 1) + 1000 = 1004
Speedup = 1304 / 1004 = 1.30 (approximately)
The (5 - 1) term is the time to fill the pipeline before the first instruction finishes. This conflict is called a structural hazard, explained fully in Pipeline Hazards. Problem 2 is exactly why pipelined processors use split L1 caches.
Problem 3: Bandwidth demand
A 1 GHz processor wants to complete one instruction per cycle. Each instruction is 4 bytes, and 30% of instructions also read or write 4 bytes of data. (a) What memory bandwidth does it need? (b) If a single shared path can deliver only 4 GB/s, what is the best possible instruction rate?
(a) Instruction traffic = 10^9 instr/s x 4 B = 4.0 GB/s
Data traffic = 0.3 x 10^9 x 4 B = 1.2 GB/s
Total = 5.2 GB/s
(b) Bytes needed per instruction = 4 + 0.3 x 4 = 5.2 B
Max rate = 4 x 10^9 B/s / 5.2 B = 0.769 x 10^9 instr/s (about 769 MIPS)
The CPU runs at only about 77% of its peak.
With two separate 4 GB/s paths (one for instructions, one for data), instructions need exactly 4 GB/s and data only 1.2 GB/s, so the processor can run at its full 1000 MIPS.
Common Mistakes Students Make
Saying modern PCs are "pure Harvard" because they have split caches. Main memory is still unified; the correct term is modified Harvard.
Thinking Von Neumann means "no cache". The architecture is about how instructions and data share memory, not about the presence of caches.
Believing Harvard is always better. It costs more hardware, splits memory rigidly and makes loading programs as data awkward. That is why general-purpose computers keep a unified memory.
Confusing the stored-program concept with the Von Neumann architecture. Both Harvard and Von Neumann machines store programs in memory; the difference is whether that memory is shared with data.
Assuming both architectures share one address space. In a Harvard machine, instruction address 0 and data address 0 are different locations.
GATE / Interview Questions
1. What is the stored-program concept?
Instructions are stored in memory as binary numbers, just like data, and the CPU fetches and executes them from there. Changing the program only requires loading different contents into memory, not rewiring hardware.
2. What is the Von Neumann bottleneck and how do modern computers reduce it?
It is the limit on throughput caused by the single shared path between CPU and memory, over which both instructions and data must travel. It is reduced by cache hierarchies, split L1 instruction and data caches, wider buses, prefetching and keeping more data in registers.
3. Why do DSPs and microcontrollers commonly use Harvard architecture?
DSPs need to fetch an instruction and one or more operands every cycle for signal-processing loops, which a single bus cannot supply. Microcontrollers naturally keep programs in on-chip flash and variables in SRAM, so separate buses are simple and give predictable timing.
4. A processor has split L1 instruction and data caches but a single main memory. Which architecture is it?
Modified Harvard: Harvard-style parallel access at the L1 level, Von Neumann-style unified memory and address space below it.
5. Why is self-modifying code difficult on a Harvard machine?
The program can only write to data memory, but instructions are fetched from a separate instruction memory, so newly written bytes are never executed unless special hardware or instructions copy them across.
6. In a pipelined CPU with one memory port, 25% of instructions are loads/stores and each causes a 1-cycle fetch stall. What is the CPI if the ideal CPI is 1?
CPI = 1 + 0.25 × 1 = 1.25. With separate instruction and data memories (or split L1 caches) the stall disappears and CPI = 1.
Summary
The stored-program concept keeps instructions in memory as binary data, so the same hardware can run any program.
Von Neumann: one memory and one bus for instructions and data. Simple and flexible, but limited by the Von Neumann bottleneck.
Harvard: separate instruction and data memories and buses, allowing simultaneous fetch and data access; used in microcontrollers and DSPs.
Modified Harvard: unified main memory with split L1 instruction/data caches (or separate code/data buses), used by nearly all modern processors.
Split paths remove the memory structural hazard in pipelines: with 30% loads/stores, CPI falls from 1.3 to 1.0 in our example.
Everything stored in these memories, whether instructions or data, is a pattern of bits. In the next chapter, Number Systems & Conversions, we learn how numbers are written in binary, octal and hexadecimal and how to convert between them.