Chapter 01 of 39

Introduction to Computer Organization & Architecture

You write c = a + b; in C, press compile, and a moment later the program runs. Somewhere between that line and the answer on the screen, a chip made of billions of tiny switches fetched instructions from memory, decoded them, moved numbers between registers, added them in a circuit and wrote the result back. Computer Organization and Architecture (COA) is the subject that explains exactly how that happens.

In this chapter we will see what "architecture" and "organization" mean (and why they are different), the five functional units of every computer, the registers the processor uses to run one instruction, the layers between your program and the hardware, and how to measure a computer's performance with the CPU time equation and Amdahl's law. The performance formulas appear in GATE almost every year, so we will practise them with solved problems.


Architecture vs Organization

The two words in the subject's name are often used together, but they mean different things.

Computer architecture refers to the attributes of a computer that are visible to a programmer (or compiler) and that directly affect how a program behaves: the instruction set, the number and size of registers, the data types and their sizes, the addressing modes and the way I/O is done.

Computer organization refers to how those architectural features are actually implemented in hardware: the datapath, control signals, the kind of memory used, cache sizes, how many adders there are, the clock speed and so on. These details are usually invisible to the programmer.

A simple way to remember it: architecture is what the machine does (the contract), organization is how it does it (the implementation).

Classic example. Whether a processor has a MUL (multiply) instruction is an architectural decision: a programmer can see it and use it. Whether that multiplication is done by a fast dedicated multiplier circuit, or by the adder doing repeated shift-and-add steps over several clock cycles, is an organizational decision. Both machines run the same program and get the same answer; one is just faster or cheaper.

This separation is very powerful. Intel and AMD processors are built very differently inside (different pipelines, caches, branch predictors), yet both run the same x86-64 programs because they share the same architecture. Back in 1964, IBM's System/360 was the first famous family to use this idea: cheap and expensive models shared one architecture, so customers could upgrade without rewriting software.

Aspect

Architecture

Organization

Question it answers

What does the computer do?

How does it do it?

Visible to programmer?

Yes

Mostly no (affects speed, not correctness)

Examples

Instruction set, register count, word size, addressing modes, data formats

Pipeline depth, cache size, control unit design, bus width, memory technology

Changes how often?

Rarely (must stay compatible for years)

With every new chip generation

Real example

x86-64, ARMv8-A, RISC-V

A specific core design implementing one of these

The part of the architecture that forms the contract between software and hardware is called the Instruction Set Architecture (ISA). We will study it in detail in Instruction Set Architecture: RISC vs CISC. The internal design of a particular processor implementing an ISA is often called its microarchitecture.


Layers Between Your Program and the Hardware

No one designs a computer by thinking about billions of transistors at once. Computers are built as layers, each hiding the details of the layer below.

  Application            (browser, game, your C program)
        |
  High-level language    c = a + b;
        |   compiler
  Assembly language      lw  t0, 0(s0)
        |                lw  t1, 4(s0)
        |                add t2, t0, t1
        |   assembler
  Machine code           0000000 00110 00101 000 00111 0110011   (the "add" above, RISC-V)
        |
  ISA  -------------------- the hardware/software contract --------------------
        |
  Microarchitecture      datapath, pipeline, caches, control unit
        |
  Digital logic          gates, adders, multiplexers, flip-flops
        |
  Electronic devices     transistors

COA lives mainly in the middle of this picture: from the ISA down to digital logic. A compiler writer needs the layers above the ISA; a chip designer works below it. A good engineer understands both sides, because many performance problems (cache misses, branch mispredictions, slow integer division) only make sense when you know what the hardware is doing.


The Five Functional Units

Every general-purpose computer, from a microcontroller in a washing machine to a server, is built from the same five functional units.

                 +-----------------------------------------+
                 |              Processor (CPU)            |
   +-------+     |   +---------------+   +--------------+  |     +--------+
   | Input |---->|   |  Control Unit |   |     ALU      |  |---->| Output |
   +-------+     |   +---------------+   +--------------+  |     +--------+
                 |           |         Registers           |
                 +-----------|-----------------------------+
                             |   (instructions and data)
                             v
                 +-----------------------------------------+
                 |                Memory                   |
                 +-----------------------------------------+

Unit

Job

Examples

Input

Brings data and programs into the computer

Keyboard, mouse, camera, network card, sensors

Memory

Stores programs and data

Registers, cache, RAM (primary); SSD, hard disk (secondary)

Arithmetic and Logic Unit (ALU)

Performs arithmetic (add, subtract, compare) and logic (AND, OR, NOT, shifts)

Integer adder, shifter, multiplier inside the CPU

Output

Sends results to the outside world

Display, printer, speaker, network card

Control Unit (CU)

Fetches and decodes instructions and generates the timing and control signals that tell every other unit what to do and when

Hardwired or microprogrammed control logic

The ALU, the control unit and a set of registers together form the processor or Central Processing Unit (CPU). Input and output units together are called I/O. The units talk to each other over wires called buses, which we will study in Bus Structure & Bus Arbitration.

Why separate the control unit from the ALU? Because the ALU only knows how to compute; it has no idea what to compute next. The control unit is like a conductor reading the score (the program) and telling each musician (ALU, registers, memory) when to play.

Primary vs Secondary Memory

  • Primary (main) memory is RAM. The CPU can read or write any byte directly using its address. It is fast (tens of nanoseconds) but volatile: contents are lost when power goes off.

  • Secondary memory (SSD, hard disk) is much larger and cheaper per byte and is non-volatile, but far slower (microseconds to milliseconds). The CPU cannot execute a program directly from it; the program is first loaded into main memory.

  • Cache is a small, very fast memory between the CPU and main memory that keeps recently used data close to the processor. A whole group of chapters, starting with Memory Hierarchy, is devoted to it.


Inside the Processor: Key Registers

A register is a small, very fast storage location inside the CPU, typically one word wide (for example 32 or 64 bits). Some registers are visible to the programmer; others are used internally by the hardware to run each instruction.

Register

Full name

What it holds

PC

Program Counter

The memory address of the next instruction to fetch

IR

Instruction Register

The instruction currently being decoded and executed

MAR

Memory Address Register

The address the CPU is about to read from or write to

MDR (also MBR)

Memory Data (Buffer) Register

The data just read from memory, or about to be written to memory

R0 ... R(n-1)

General-purpose registers

Operands and results used by the program

Status / flags

Program Status Word (PSW) or condition codes

Bits such as Zero, Carry, Negative and Overflow set by the last ALU operation

MAR and MDR are the CPU's "doorway" to memory: the address goes out through MAR, data comes in or goes out through MDR. The flags are explained in Binary Arithmetic & Overflow.

How One Instruction Runs

Consider a program with these two instructions stored in memory:

Load  R2, LOC        ; R2 <- contents of memory location LOC
Add   R4, R2, R3     ; R4 <- R2 + R3

Here is what the processor does for the Load, step by step:

  1. Fetch: copy PC into MAR and request a memory read. The instruction arrives in MDR and is copied into IR. PC is incremented to point to the next instruction (by 4 if instructions are 4 bytes long).

  2. Decode: the control unit examines the bits in IR and finds out it is a Load with destination R2 and address LOC.

  3. Execute / memory access: LOC is placed in MAR and a read is requested. The word arrives in MDR.

  4. Write back: MDR is copied into R2.

The Add is fetched and decoded the same way, but its execute step needs no memory access: R2 and R3 are sent to the ALU, and the sum is written into R4.

This repeat-forever loop of fetch, decode, execute is the heartbeat of every processor. It is called the instruction cycle, and it gets its own chapter, Instruction Format & Instruction Cycle. The idea that instructions sit in the same memory as data and are fetched one by one is the stored-program concept, which is the topic of the next chapter.


Words, Bytes and Addresses

Memory is organised as a long array of storage cells, each with an address. Some terms you will see in every COA problem:

  • Bit: a single 0 or 1. Byte: 8 bits. Nibble: 4 bits.

  • Word: the natural unit of data the processor handles at once, usually the register width (32 bits on a 32-bit machine, 64 bits on a 64-bit one).

  • Byte-addressable memory: every byte has its own address. Almost all modern computers are byte-addressable.

  • Word-addressable memory: each address points to a whole word.

With k address bits we can name 2k different locations. In COA problems memory sizes use powers of two: 1 K = 210, 1 M = 220, 1 G = 230. (The standards bodies call these KiB, MiB and GiB to avoid confusion with the decimal kilo = 1000, which is what disk makers and network speeds use.)

Solved problem. A computer has 4 GB of main memory. How many address bits are needed if it is (a) byte-addressable, (b) word-addressable with 32-bit words?

  • 4 GB = 22 × 230 = 232 bytes. (a) One address per byte, so 32 bits.

  • A 32-bit word is 4 = 22 bytes, so there are 232 / 22 = 230 words. (b) 30 bits.

Endianness

When a 4-byte word such as 0x12345678 is stored in byte-addressable memory, which byte goes at the lowest address?

  • Big-endian: the most significant byte (0x12) at the lowest address. Used in network protocols ("network byte order").

  • Little-endian: the least significant byte (0x78) at the lowest address. Used by x86 and, by default, by ARM and RISC-V.

You can see it yourself with this C program:

#include <stdio.h>
#include <stdint.h>

int main(void) {
    uint32_t word = 0x12345678;
    unsigned char *p = (unsigned char *)&word;   /* look at the 4 bytes one by one */
    for (int i = 0; i < 4; i++)
        printf("address base+%d : 0x%02X\n", i, p[i]);
    printf("sizeof(int) = %zu, sizeof(long) = %zu, sizeof(void *) = %zu\n",
           sizeof(int), sizeof(long), sizeof(void *));
    return 0;
}

Compiled with GCC on a 64-bit x86 Linux machine, it prints:

address base+0 : 0x78
address base+1 : 0x56
address base+2 : 0x34
address base+3 : 0x12
sizeof(int) = 4, sizeof(long) = 8, sizeof(void *) = 8

The lowest address holds 0x78, so x86 is little-endian. Notice also that pointers are 8 bytes: the machine uses 64-bit addresses.


A Very Short History

Generation

Technology

Landmarks

First (1940s to mid 1950s)

Vacuum tubes

ENIAC (1945); EDVAC design report (1945) describing the stored-program idea

Second (late 1950s to mid 1960s)

Transistors

Smaller, cheaper, more reliable machines; high-level languages such as FORTRAN and COBOL

Third (mid 1960s to 1970s)

Integrated circuits (ICs)

IBM System/360 family (1964); many transistors on one chip

Fourth (1970s onward)

VLSI, microprocessors

Intel 4004 (1971), the first commercial single-chip microprocessor; personal computers

Today

Billions of transistors per chip

Multicore CPUs, GPUs, systems-on-chip in phones

In 1965 Gordon Moore observed that the number of transistors on a chip was doubling regularly; in 1975 he revised the rate to about every two years. This trend, called Moore's law, held for decades. Around the mid-2000s, however, clock speeds stopped rising quickly because chips were getting too hot (the "power wall"). Designers started using the extra transistors to put several cores on one chip instead, which is why your laptop has a multicore processor. We return to this in Multicore Processors & Cache Coherence.


Measuring Performance

"Computer A is faster than computer B" can mean two different things:

  • Response time (execution time, latency): how long one task takes from start to finish. Matters to a single user.

  • Throughput (bandwidth): how many tasks are completed per unit time. Matters to a server handling thousands of requests.

For one program, we define performance = 1 / execution time. If A is n times faster than B, then

Performance_A / Performance_B = ExecutionTime_B / ExecutionTime_A = n

The CPU Time Equation

A processor is driven by a clock. One clock period (cycle time) T is the inverse of the clock rate f: a 2.5 GHz clock has T = 1 / (2.5 × 109) s = 0.4 ns.

Different instructions may take different numbers of cycles, so we use the average Cycles Per Instruction (CPI). Then:

CPU time = Instruction count (IC) × CPI × Clock cycle time = (IC × CPI) / Clock rate

This is the single most important formula in this subject. Each factor is affected by different things:

Factor

Mainly affected by

Instruction count

The algorithm, the programming language, the compiler, the ISA

CPI

The ISA, the organization (pipelining, caches), the compiler's choice of instructions

Clock cycle time

The organization (how much work per cycle) and the chip technology

When instructions fall into classes with different CPIs, the average is a weighted sum:

CPI = sum over classes of ( fraction of instructions in class i ) x CPI_i

MIPS

MIPS (Million Instructions Per Second) is a popular but dangerous metric:

MIPS = Instruction count / (Execution time x 10^6) = Clock rate / (CPI x 10^6)

It is dangerous because it ignores the instruction count. A machine (or compiler) can show a higher MIPS rating while actually running the program more slowly, as Problem 4 below shows. Use execution time to compare computers; use MIPS only when IC is the same.

Amdahl's Law

Suppose you speed up only a part of a task. How much faster does the whole task get?

Amdahl's law: if a fraction f of the original execution time is sped up by a factor s, the overall speedup is
Speedup = 1 / ( (1 - f) + f / s )

The intuition: the part you did not improve, (1 - f), stays exactly as slow as before, and it eventually dominates. Even if s becomes infinite, the speedup can never exceed 1 / (1 - f). In plain words: make the common case fast, because improving a rare case barely helps.


Solved Problems on Performance

Problem 1: Basic CPU time

A program executes 5 × 108 instructions on a 2 GHz processor with an average CPI of 1.6. Find the CPU time and the MIPS rating.

CPU time = IC x CPI / f = (5 x 10^8 x 1.6) / (2 x 10^9) = 8 x 10^8 / 2 x 10^9 = 0.4 s
MIPS     = f / (CPI x 10^6) = 2 x 10^9 / (1.6 x 10^6) = 1250

Check: 5 × 108 instructions in 0.4 s is 1.25 × 109 instructions per second = 1250 million per second. Both methods agree.

Problem 2: CPI from an instruction mix

A 2.5 GHz processor runs a program of 109 instructions with this mix:

Class

Fraction

CPI

Contribution

ALU

50%

1

0.50

Load

20%

3

0.60

Store

10%

2

0.20

Branch

20%

2

0.40

Total

100%

CPI = 1.70

CPU time = 10^9 x 1.7 / (2.5 x 10^9) = 0.68 s

Notice that loads are only 20% of instructions but contribute 0.6 of the 1.7 CPI. Speeding up loads (with a better cache, for example) would help a lot.

Problem 3: Which machine is faster?

Machine A has a 3 GHz clock and CPI 2.0 for a program. Machine B, with the same ISA, has a 2 GHz clock and CPI 1.2 for the same program. Which is faster, and by how much?

The instruction count is the same, so compare time per instruction = CPI / f:

A: 2.0 / (3 x 10^9) = 0.667 ns per instruction
B: 1.2 / (2 x 10^9) = 0.600 ns per instruction
Speedup of B over A = 0.667 / 0.600 = 1.11

B is about 1.11 times faster, even though its clock is slower. Clock rate alone never tells the full story.

Problem 4: MIPS can lie

A 1 GHz machine has three instruction classes with CPI 1, 2 and 3. Two compilers produce code for the same program:

Compiler

Class 1 (CPI 1)

Class 2 (CPI 2)

Class 3 (CPI 3)

Compiler 1

4 million

2 million

1 million

Compiler 2

12 million

1 million

1 million

Compiler 1: cycles = 4x1 + 2x2 + 1x3 = 11 million, IC = 7 million
            time = 11 x 10^6 / 10^9 = 11 ms
            MIPS = 7 x 10^6 / (0.011 x 10^6) = 636 MIPS
Compiler 2: cycles = 12x1 + 1x2 + 1x3 = 17 million, IC = 14 million
            time = 17 x 10^6 / 10^9 = 17 ms
            MIPS = 14 x 10^6 / (0.017 x 10^6) = 824 MIPS

Compiler 2's code has the higher MIPS but takes longer (17 ms vs 11 ms), because it executes twice as many instructions. Execution time is the only fully reliable measure.

Problem 5: Amdahl's law

(a) 40% of a program's execution time is spent in floating-point work. A new FPU makes that part 10 times faster. What is the overall speedup?

Speedup = 1 / ( 0.6 + 0.4/10 ) = 1 / 0.64 = 1.5625
Upper limit (infinitely fast FPU) = 1 / 0.6 = 1.667

(b) 90% of the time can be enhanced. By what factor must it be sped up to get an overall speedup of 5?

5 = 1 / ( 0.1 + 0.9/s )   =>   0.1 + 0.9/s = 0.2   =>   0.9/s = 0.1   =>   s = 9

A 9 times faster enhanced part gives only a 5 times faster program. And no matter how large s becomes, the overall speedup stays below 1 / 0.1 = 10.


Common Mistakes Students Make

  • Mixing up architecture and organization. If the programmer can see it (an instruction, a register, a data size), it is architecture. If it only changes speed or cost, it is organization.

  • Comparing processors by clock rate alone. CPU time depends on IC × CPI × T; a slower clock with a lower CPI can win.

  • Using MIPS to compare different ISAs or compilers. MIPS ignores how many instructions are needed.

  • Applying Amdahl's fraction to the wrong thing. f is the fraction of execution time that is improved, not the fraction of instructions.

  • Using 1 K = 1000 in memory-address problems. In COA memory sizes, K = 210, M = 220, G = 230. (Clock rates are decimal: 1 GHz = 109 Hz.)

  • Forgetting that PC holds the address of the next instruction, not the current one (it is incremented during fetch).


GATE / Interview Questions

1. Is the presence of a hardware multiplier an architectural or organizational feature?

Organizational. The existence of a multiply instruction is architectural, but whether it is executed by a dedicated multiplier or by repeated add-and-shift is an implementation (organization) choice invisible to the programmer.

2. What are MAR and MDR used for?

MAR holds the address of the memory location to be read or written. MDR holds the data read from memory or the data to be written to memory. Together they form the CPU's interface to memory.

3. A program has 2 × 106 instructions and runs in 2 ms on a 1.5 GHz CPU. What is its average CPI?

Cycles = time × f = 2 × 10-3 × 1.5 × 109 = 3 × 106. CPI = 3 × 106 / 2 × 106 = 1.5.

4. A memory of 64 K words, each 16 bits, is word-addressable. How many address bits are needed? What if it were byte-addressable?

64 K = 216 words, so 16 bits when word-addressable. Each word is 2 bytes, so there are 217 bytes and 17 bits are needed when byte-addressable.

5. 30% of a task cannot be parallelised. What is the maximum speedup with unlimited processors?

By Amdahl's law with f = 0.7 and s → ∞, speedup → 1 / 0.3 ≈ 3.33.

6. Why did processors move to multiple cores instead of higher clock speeds?

Power and heat grow quickly with clock rate (and the voltage needed for it). Around the mid-2000s chips hit the power wall, so designers used the growing transistor budget to add more cores, gaining throughput through parallelism instead.


Summary

  • Architecture = what the programmer sees (ISA, registers, data types, addressing). Organization = how it is implemented (datapath, control, caches, memory technology).

  • Every computer has five functional units: input, memory, ALU, output and control. ALU + control + registers form the CPU.

  • Key CPU registers: PC (next instruction address), IR (current instruction), MAR (memory address), MDR (memory data), general-purpose registers and flags.

  • Instructions are executed by the repeating fetch, decode, execute cycle.

  • With k address bits we address 2k locations. Byte order in a word is big-endian or little-endian.

  • CPU time = IC × CPI / clock rate. MIPS can mislead. Amdahl's law: speedup = 1 / ((1 - f) + f/s), limited by the unimproved part.

Next, we look at the big idea that makes all of this possible, storing programs in memory, and the two classic ways to connect that memory to the CPU in Von Neumann vs Harvard Architecture.