The last two chapters, Black-Box Testing and White-Box Testing & Cyclomatic Complexity, were about how to design test cases. This chapter is about where and when they are applied. A large system is not tested in one go. It is tested in stages, starting with the smallest pieces and ending with the real users. These stages are called the levels of testing.
Knowing the levels (unit, integration, system and acceptance), the integration strategies (top-down, bottom-up, sandwich, big bang), stubs and drivers, and alpha vs beta testing is essential for university exams and very common in interviews.
A test level is a group of test activities that are organised and managed together, each focused on a particular part of the system (a unit, a group of interacting units, the whole system) and each with its own objectives, test basis, typical defects and responsible people.
The Intuition: Building a Car
A car manufacturer does not assemble a whole car and then turn the key to see whether anything works. Instead:
Each part is checked on its own: the brake pad, the spark plug, the headlight bulb. (Unit testing)
Parts are fitted together and their connections are checked: does the brake pedal actually move the brake pads? (Integration testing)
The complete car is tested on a track: speed, mileage, safety, noise. (System testing)
The customer takes a test drive and decides whether to buy. (Acceptance testing)
Each level catches a different kind of problem, and a problem found early is much cheaper to fix than one found by the customer. This matches the V-Model from V-Model, where each development phase on the left arm has a matching test level on the right arm:
Requirements analysis ◄──────────────────────► Acceptance testing
System design ◄─────────────────► System testing
Architecture design ◄──────────► Integration testing
Module design ◄─────► Unit testing
Coding
The ISTQB Foundation syllabus (v4.0) uses five level names: component (unit) testing, component integration testing, system testing, system integration testing (the system with external systems and services) and acceptance testing. Most textbooks and university syllabi use the four classic levels below.
Level 1: Unit Testing
Unit testing (component or module testing) tests the smallest separately testable pieces of software, such as a function, method or class, in isolation from the rest of the system.
Done by: usually the developer who wrote the code, often automated and run on every build.
Test basis: detailed (module) design and the code itself.
Techniques: both white-box (statement and branch coverage) and black-box (ECP, BVA).
Typical defects: wrong calculations, wrong conditions, off-by-one errors, incorrect handling of invalid input, unhandled exceptions.
Pressman lists what unit tests should examine: the module interface (are parameters passed correctly?), local data structures, boundary conditions, independent paths and error-handling paths.
Stubs and drivers
A unit rarely works alone: it is called by some modules and calls others. To test it in isolation, we replace its neighbours with simple stand-ins.
┌─────────────┐
│ DRIVER │ stands in for the CALLER (the module above):
└──────┬──────┘ sets up inputs, calls the unit, checks results
│ calls
┌──────▼──────┐
│ UNIT UNDER │
│ TEST │
└──────┬──────┘
│ calls
┌──────▼──────┐
│ STUB │ stands in for a CALLED module (the module below):
└─────────────┘ returns a fixed, simple answer
| Driver | Stub | |
|---|---|---|
| Replaces | A calling (higher-level) module | A called (lower-level) module |
| Job | Passes test inputs to the unit and collects/checks output | Accepts the call and returns a canned response |
| Needed mostly in | Bottom-up integration | Top-down integration |
| Everyday example | A test framework or main() written just for testing | A fake payment gateway that always says "success" |
Modern testing vocabulary (from Gerard Meszaros' xUnit Test Patterns) calls all such stand-ins test doubles: a dummy is passed but never used; a stub returns canned answers; a spy also records how it was called; a mock is pre-programmed with expectations about the calls it should receive; a fake has a working but simplified implementation (for example an in-memory database).
Example: unit testing with a stub
A college fee module depends on a payment gateway that the other team has not finished yet. We can still unit-test our module by stubbing the gateway with Python's unittest.mock. The test class itself plays the role of the driver.
import unittest
from unittest.mock import Mock
# ---- unit under test -------------------------------------------------
class FeeService:
def __init__(self, gateway):
self.gateway = gateway # lower-level module (not ready yet)
def pay(self, student_id, amount):
if amount <= 0:
raise ValueError("amount must be positive")
ok = self.gateway.charge(student_id, amount)
return "PAID" if ok else "FAILED"
# ---- unit tests: the test class acts as the DRIVER --------------------
class TestFeeService(unittest.TestCase):
def test_successful_payment(self):
gateway = Mock() # STUB for the payment gateway
gateway.charge.return_value = True # canned answer
service = FeeService(gateway)
self.assertEqual(service.pay("21CS042", 45000), "PAID")
gateway.charge.assert_called_once_with("21CS042", 45000)
def test_declined_payment(self):
gateway = Mock()
gateway.charge.return_value = False
self.assertEqual(FeeService(gateway).pay("21CS042", 45000), "FAILED")
def test_invalid_amount_never_reaches_gateway(self):
gateway = Mock()
with self.assertRaises(ValueError):
FeeService(gateway).pay("21CS042", 0)
gateway.charge.assert_not_called()
if __name__ == "__main__":
unittest.main(verbosity=2)
Output (Python 3.14):
test_declined_payment (__main__.TestFeeService.test_declined_payment) ... ok
test_invalid_amount_never_reaches_gateway (__main__.TestFeeService.test_invalid_amount_never_reaches_gateway) ... ok
test_successful_payment (__main__.TestFeeService.test_successful_payment) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK
Notice that assert_called_once_with and assert_not_called check how the gateway was used, which makes this double a mock in Meszaros' terms. The tests run in a millisecond, need no network and no real money, and can be run hundreds of times a day.
Level 2: Integration Testing
Integration testing tests the interfaces and interactions between integrated units (or between subsystems) to expose defects that appear only when they work together.
"Each unit passed its unit tests, so the system must work" is a dangerous assumption. Typical integration defects:
Data lost or misinterpreted across an interface (rupees vs paise, date formats DD-MM vs MM-DD, a field renamed in one module only).
Wrong order or number of parameters; wrong assumptions about who validates input.
One module's side effect breaking another (shared global data, which is common coupling from Modularity, Coupling & Cohesion).
Timing and resource problems when modules run together.
Integration strategies
Consider this module hierarchy (a structure chart) for a college portal:
M1 Main menu
┌──────────┼──────────┐
M2 M3 M4
Admission Results Fees
┌───┴───┐ │
M5 M6 M7
Verify Allot Payment
docs roll no. gateway
Big-bang integration
All modules are combined at once and the whole thing is tested. It needs no stubs or drivers, but when a test fails, the defect could be in any module or any interface, so fault localisation is very hard. It is only reasonable for very small systems.
Incremental integration: top-down
Start with the top module M1, using stubs for M2, M3 and M4. Then replace stubs with real modules one at a time, testing after each addition.
Depth-first order: M1, M2, M5, M6, M3, M4, M7 (finish one complete branch first; useful to get a full feature such as "Admission" working early).
Breadth-first order: M1, M2, M3, M4, M5, M6, M7 (level by level).
Advantages: the main control logic and user-visible flows are tested early, and an early skeleton version can be demonstrated. Disadvantages: many stubs are needed (here a stub for each of M2–M7 at some stage), and stubs that return fixed values can make lower-level processing look correct when it is not, so testing of the low-level modules happens late.
Incremental integration: bottom-up
Start with the lowest-level (leaf) modules M5, M6 and M7. Combine them into clusters (also called builds), and test each cluster using a driver that plays the role of the caller. Then replace drivers with real modules moving upwards:
Step 1: [M5 + M6] tested with a driver for M2 [M7] tested with a driver for M4
Step 2: M2 + (M5, M6) M4 + (M7) M3 each tested with a driver for M1
Step 3: M1 + everything → complete system
Advantages: low-level modules, often the ones doing real computation, I/O or hardware access, are tested thoroughly and early; no stubs are needed. Disadvantages: drivers are needed, and the program as a whole does not exist until the last module is added, so major design flaws in the top-level control may be found late.
Sandwich (hybrid or mixed) integration
Top-down from the top and bottom-up from the bottom at the same time, meeting at a middle (target) layer. It combines the advantages of both and suits large systems with several teams, but needs both stubs and drivers and careful coordination.
Comparison of strategies
| Aspect | Big bang | Top-down | Bottom-up | Sandwich |
|---|---|---|---|---|
| Stubs needed | No | Yes | No | Yes |
| Drivers needed | No | No (the top module is driven by the tester or UI) | Yes | Yes |
| Fault localisation | Very difficult | Easy | Easy | Easy |
| Early working skeleton | No | Yes | No | Partly |
| Low-level modules tested early | No | No | Yes | Yes |
| Best for | Tiny systems | Systems where control flow and UI are critical | Systems with critical low-level components | Large, layered systems |
In modern projects with continuous integration (see DevOps & CI/CD), integration happens many times a day: each small change is merged and the automated integration tests run immediately. This is incremental integration taken to its natural limit.
Level 3: System Testing
System testing tests the complete, integrated system against its specified requirements, both functional and non-functional, in an environment as close to production as possible.
Done by: usually an independent test team (not the developers).
Test basis: the SRS (see Software Requirements Specification (SRS)), use cases, system design.
Techniques: mainly black-box.
Types of system testing
| Type | Question it answers | Example (college portal) |
|---|---|---|
| Functional | Does every feature in the SRS work end to end? | Admission → roll number → fee payment → receipt |
| Performance | Are response times and throughput acceptable under expected load? | Result page loads within 2 s for 1,000 users |
| Load | How does it behave at the expected peak load? | 20,000 students checking results at 10 a.m. on result day |
| Stress | What happens beyond the limit? Does it fail gracefully and recover? | 3× the peak load; does it show a "please retry" page or crash and lose data? |
| Spike | Can it handle a sudden jump in load? | Load jumps from 500 to 20,000 users in one minute |
| Endurance (soak) | Does it degrade over a long period (memory leaks, log growth)? | Normal load for 48 hours continuously |
| Volume | Does it handle large amounts of data? | A database with 10 years of student records |
| Security | Is it protected against unauthorised access and attacks? | SQL injection in the login form; students viewing others' marks |
| Usability | Is it easy and pleasant to use? | Can a first-year student pay fees without help? |
| Recovery | Does it recover correctly after crashes or failures? | Server restart in the middle of a payment |
| Compatibility | Does it work across browsers, devices, OS versions? | Chrome, Firefox, Safari; Android and iOS |
| Installation / configuration | Can it be installed, upgraded, uninstalled, configured correctly? | Upgrade from version 2.3 to 3.0 keeps all data |
| Accessibility | Can people with disabilities use it? | Screen reader support, keyboard-only navigation |
Load, stress, spike, endurance and volume testing are all forms of performance testing in the broad sense. The difference between load and stress testing is a classic interview question: load testing stays within expected limits; stress testing deliberately goes beyond them to find the breaking point and see how the system fails and recovers.
Level 4: Acceptance Testing
Acceptance testing is formal testing with respect to user needs, requirements and business processes, conducted to let the customer, users or other authorised entity decide whether to accept the system.
Its goal is not mainly to find defects (those should be gone by now) but to build confidence and validate fitness for use. It is the clearest example of validation ("are we building the right product?") from Software Testing Fundamentals.
Forms of acceptance testing
User acceptance testing (UAT): real users check that the system supports their actual work, e.g. the college accounts office processing a week of fee payments.
Operational acceptance testing (OAT): system administrators check backups, restore, disaster recovery, user management, monitoring and maintenance tasks.
Contractual acceptance testing: against acceptance criteria written in a contract (common for software built by outsourcing firms).
Regulatory acceptance testing: against laws and regulations, e.g. data protection rules or RBI guidelines for payment systems.
Alpha and beta testing: used for software sold to many customers, where you cannot run UAT with every customer.
Alpha vs beta testing
| Aspect | Alpha testing | Beta testing |
|---|---|---|
| Where | At the developer's site | At the users' own sites (their devices and environment) |
| Who | Potential users or an internal team (not the developers of the feature), observed by developers | A selected group of real external users |
| Environment | Controlled | Uncontrolled, real world |
| When | Before beta, near the end of development | After alpha, just before general release |
| Defects | Developers see problems immediately and can fix them quickly | Users report problems back; developers do not watch directly |
| Example | Company employees using a new app version internally | A public "beta" programme of a mobile app on the Play Store |
All Four Levels Side by Side
| Aspect | Unit | Integration | System | Acceptance |
|---|---|---|---|---|
| Object | Function, class, module | Interfaces between modules / subsystems | Whole system | Whole system in its business context |
| Test basis | Detailed design, code | Architecture, interface specifications | SRS, use cases | User needs, business processes, contract |
| Done by | Developers | Developers / testers | Independent testers | Customers, users |
| Main approach | White-box + black-box | Mostly black-box with some white-box (grey-box) | Black-box | Black-box |
| Typical defects | Logic, calculation, boundary errors | Interface, data format, communication errors | Requirement mismatches, performance, security issues | Workflow does not fit real business needs |
| Uses stubs/drivers? | Yes | Yes (incremental strategies) | No | No |
Regression Testing Across Levels
Every time code changes (a bug fix, a new feature, a library upgrade), something that used to work may break. Regression testing re-runs existing tests to catch such side effects, and it applies at every level. Because a full regression suite can be large, teams select tests by risk and by what changed, and they automate as much as possible.
The test pyramid
Mike Cohn's test pyramid gives a practical rule for how many automated tests to have at each level:
╱╲
╱ ╲ few UI / end-to-end tests
╱ UI ╲ (slow, brittle, expensive)
╱──────╲
╱ Service╲ some integration / API tests
╱──────────╲
╱ Unit ╲ many unit tests
╱──────────────╲ (fast, cheap, precise)
Unit tests run in milliseconds and pinpoint the faulty function; end-to-end tests through the user interface are slow and break whenever the UI changes. An "ice-cream cone" (mostly manual and UI tests, few unit tests) is a common anti-pattern.
Common Mistakes
Swapping stubs and drivers. Stub = replaces a module that is called (below); driver = replaces the caller (above). Top-down uses stubs; bottom-up uses drivers.
Thinking integration testing is unnecessary when unit tests pass. Units can each be correct and still disagree about an interface.
Confusing system and acceptance testing. System testing checks against the specification (verification-oriented, done by testers); acceptance testing checks fitness for the users' real needs (validation, done or witnessed by customers).
Mixing up load and stress testing. Load = expected peak; stress = beyond the limit.
Saying beta testing happens at the developer's site. Alpha is at the developer's site; beta is in the users' environment.
Treating levels as strictly sequential in Agile. In iterative and Agile development, all levels happen within each iteration, mostly automated.
Interview and Exam Questions
Q1. What is the difference between a stub and a driver? Which integration strategy needs which?
A stub replaces a lower-level module called by the unit under test and returns canned results; a driver replaces a higher-level module, calling the unit under test with test inputs. Top-down integration needs stubs; bottom-up integration needs drivers; sandwich needs both; big bang needs neither.
Q2. Why is big-bang integration discouraged?
Because all modules are combined at once, a failure could come from any module or any interface, so locating the fault is very hard. Testing also starts late, since everything must be ready first. Incremental strategies add one module at a time, so a new failure most likely involves the module just added.
Q3. In top-down integration of the hierarchy M1 → {M2, M3, M4}, M2 → {M5, M6}, M4 → {M7}, give the depth-first and breadth-first orders.
Depth-first: M1, M2, M5, M6, M3, M4, M7. Breadth-first: M1, M2, M3, M4, M5, M6, M7.
Q4. Distinguish alpha and beta testing.
Alpha testing is done at the developer's site in a controlled environment by potential users or an internal team, with developers observing. Beta testing is done at the users' own sites in a real environment by selected external users, who report problems back. Alpha comes first.
Q5. Which level of testing is done by an independent team against the SRS, and which is mainly white-box?
System testing is done by an independent test team against the SRS. Unit testing is the level where white-box techniques are used most.
Q6. A site works fine with 2,000 users but crashes with 8,000. The expected peak is 5,000. Which tests would reveal what?
A load test at 5,000 users checks whether the expected peak is handled (and whether response times are acceptable). A stress test pushes towards and past 8,000 to find the breaking point and check that the failure is graceful (no data loss, clear error message) and that the system recovers.
Q7. What is operational acceptance testing?
Acceptance testing by operations/administration staff of operational aspects: backup and restore, disaster recovery, installation and upgrade, user management, monitoring, security patching and maintenance procedures.
Summary
Testing proceeds in levels matching the V-Model: unit ↔ module design, integration ↔ architecture, system ↔ system design/SRS, acceptance ↔ user requirements.
Unit testing isolates a module using drivers (replace callers) and stubs (replace called modules); modern frameworks provide test doubles such as mocks.
Integration testing targets interfaces. Strategies: big bang (hard to debug), top-down (stubs, early skeleton, depth- or breadth-first), bottom-up (drivers, clusters), sandwich (both).
System testing checks the whole system against the SRS, including non-functional types: performance, load, stress, spike, endurance, volume, security, usability, recovery, compatibility.
Acceptance testing lets the customer decide: UAT, OAT, contractual, regulatory, and alpha (developer's site) vs beta (users' sites).
Regression testing applies at every level; the test pyramid recommends many unit tests, fewer integration tests and few UI tests.
Next, Software Quality & Metrics steps back from testing to ask what "quality" means and how we measure it.