In Software Testing Fundamentals we saw that exhaustive testing is impossible: even a function with two 32-bit integer inputs has more combinations than we could ever run. So the real skill of a tester is choosing a small set of test cases that has a high chance of finding defects. This chapter covers the first big family of techniques for doing that: black-box testing, where test cases are designed from the specification alone, without looking at the code.
These techniques (equivalence partitioning, boundary value analysis, decision tables, state transition testing and a few more) are asked in almost every university exam, appear in GATE-style numerical questions ("how many test cases does boundary value analysis need?") and are a favourite topic in QA and developer interviews.
Black-box testing (also called specification-based, behavioural or functional testing) is a test design approach in which test cases are derived from the requirements or specification of the software, treating the program as a "black box" whose internal structure is not known or not used.
The Intuition: Testing a Vending Machine
Imagine you are asked to test a vending machine, but you are not allowed to open it. What do you do? You read the label on the front: "Accepts ₹10 and ₹20 coins. Snacks cost ₹30. Returns change." Then you try things based on that label:
Insert exactly ₹30 and check that a snack comes out (normal case).
Insert ₹40 and check that ₹10 change comes back (another class of input).
Insert ₹20 only and check that nothing comes out (just below the price: a boundary).
Insert a ₹5 coin or a foreign coin and check it is rejected (invalid input).
Press "cancel" after inserting money and check you get a refund (a state change).
You never looked at the wiring, yet you designed sensible tests. Each technique in this chapter is a systematic version of one of these instincts.
inputs outputs
(from the specification) (compared with expected
results from the spec)
───────────────────────► ┌─────────────┐ ───────────────────────►
│ BLACK BOX │
│ (code is not│
│ looked at) │
└─────────────┘
What black-box testing can find well:
Missing or incorrect functions (a feature in the SRS that was never built).
Interface errors (wrong inputs accepted, wrong outputs produced).
Errors in handling boundary and invalid data.
Behaviour and performance errors visible from outside.
What it cannot do: tell you which lines of code were never executed. Code that is not described in the specification (for example a hidden debug option) may never be tested. That gap is filled by White-Box Testing & Cyclomatic Complexity, the next chapter.
Equivalence Class Partitioning (ECP)
The key observation is that a program usually treats many inputs the same way. If a grading function gives "C" for 50 marks, it almost certainly uses the same code path for 51 or 55. Testing all of them wastes effort.
An equivalence class (or partition) is a set of input (or output) values that the specification says should be processed in the same way. Equivalence partitioning divides the input domain into such classes and picks one representative value from each class, assuming that if one value in a class finds a defect, the others would too.
Guidelines for forming classes
| If the input condition is… | Valid classes | Invalid classes |
|---|---|---|
| A range, e.g. age 18 to 60 | 1: 18 ≤ age ≤ 60 | 2: age < 18, age > 60 |
| A specific value, e.g. PIN length is exactly 4 | 1: length = 4 | 2: length < 4, length > 4 |
| A set of values, e.g. state code in {MH, KA, TN} | 1 per value (if treated differently) or 1 for the set | 1: any value outside the set |
| A boolean / "must be" condition, e.g. first character must be a letter | 1: starts with a letter | 1: does not start with a letter |
Do not forget the invalid classes. Many real defects are about what happens when the input is wrong: a crash, a silent wrong answer, or a confusing error message.
Worked example: a grading function
Specification: grade(marks) takes an integer from 0 to 100. Marks 75–100 get "A", 60–74 get "B", 40–59 get "C" and 0–39 get "F". Any value outside 0–100 must be rejected with an error.
Step 1: identify classes.
| Class | Range | Type | Representative | Expected |
|---|---|---|---|---|
| E1 | marks < 0 | Invalid | −5 | Error |
| E2 | 0–39 | Valid | 20 | F |
| E3 | 40–59 | Valid | 50 | C |
| E4 | 60–74 | Valid | 67 | B |
| E5 | 75–100 | Valid | 88 | A |
| E6 | marks > 100 | Invalid | 120 | Error |
Step 2: result. Six test cases replace 101 valid values plus infinitely many invalid ones. (If the input could also be a non-integer such as "abc" or 45.5, those would be further invalid classes.)
ECP with several inputs
When there are several inputs, each with its own classes, there are two common ways to combine them (terminology from Jorgensen's Software Testing: A Craftsman's Approach):
Weak equivalence testing: every class of every input is used at least once. Number of tests = the largest number of classes of any single input.
Strong equivalence testing: every combination of classes is tested (the Cartesian product). Number of tests = product of the class counts.
"Normal" variants use only valid classes; "robust" variants add invalid classes. Example: input x has 3 valid classes and input y has 2. Weak normal needs max(3, 2) = 3 tests; strong normal needs 3 × 2 = 6.
A practical rule when adding invalid classes: put only one invalid value per test case. If a test has two invalid inputs and the program reports an error, you cannot tell which check actually worked; one error may be masking the other.
Boundary Value Analysis (BVA)
Experience shows that defects crowd at the edges of classes. Programmers write > where they meant >=, loop one time too many, or forget that 100 is a valid mark. This is the famous "off-by-one" error.
Boundary value analysis is a technique that selects test values at and around the boundaries of equivalence classes, because faults are more likely there than in the middle of a class.
Which values to pick
For a range [min, max], classical BVA (as used in most Indian textbooks and GATE questions) picks five values for the variable:
min min+1 nominal max−1 max
│ │ │ │ │
────●─────────●─────────────●──────────────●─────────●────
└───── values inside the valid range ───────────────┘
Robustness testing adds two invalid values: min−1 and max+1
The ISTQB syllabus (Foundation Level v4.0) describes the same idea as 2-value BVA (each boundary value and its closest neighbour in the adjacent class) and 3-value BVA (the boundary value and both its neighbours). The names differ, but the principle is identical: test where one class ends and the next begins.
Worked example: the grading function again
The boundaries are between the classes: −1|0, 39|40, 59|60, 74|75 and 100|101. Using 2-value BVA on each boundary gives:
| Boundary | Test values | Expected |
|---|---|---|
| Lower limit of input | −1, 0 | Error, F |
| F / C | 39, 40 | F, C |
| C / B | 59, 60 | C, B |
| B / A | 74, 75 | B, A |
| Upper limit of input | 100, 101 | A, Error |
Seeing BVA catch a bug that ECP misses
Here is an implementation with a classic off-by-one fault: it uses marks > 60 instead of marks >= 60. We run both the ECP tests and the BVA tests using Python's unittest:
import unittest
def grade(marks):
if marks < 0 or marks > 100:
raise ValueError("marks must be between 0 and 100")
if marks >= 75:
return "A"
if marks > 60: # fault: should be marks >= 60
return "B"
if marks >= 40:
return "C"
return "F"
class EquivalenceTests(unittest.TestCase):
def test_partitions(self):
for marks, expected in [(20, "F"), (50, "C"), (67, "B"), (88, "A")]:
with self.subTest(marks=marks):
self.assertEqual(grade(marks), expected)
def test_invalid(self):
for marks in (-5, 120):
with self.subTest(marks=marks):
self.assertRaises(ValueError, grade, marks)
class BoundaryTests(unittest.TestCase):
def test_boundaries(self):
cases = [(0, "F"), (39, "F"), (40, "C"), (59, "C"),
(60, "B"), (74, "B"), (75, "A"), (100, "A")]
for marks, expected in cases:
with self.subTest(marks=marks):
self.assertEqual(grade(marks), expected)
def test_invalid_boundaries(self):
for marks in (-1, 101):
with self.subTest(marks=marks):
self.assertRaises(ValueError, grade, marks)
if __name__ == "__main__":
unittest.main()
Output (Python 3.14):
F...
======================================================================
FAIL: test_boundaries (__main__.BoundaryTests.test_boundaries) (marks=60)
----------------------------------------------------------------------
Traceback (most recent call last):
File "grade_test.py", line 30, in test_boundaries
self.assertEqual(grade(marks), expected)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'C' != 'B'
- C
+ B
----------------------------------------------------------------------
Ran 4 tests in 0.001s
FAILED (failures=1)
Both ECP test methods passed: 67 is comfortably inside the "B" class, so the wrong operator never mattered. Only the boundary value 60 exposed the fault. This is exactly why ECP and BVA are used together: ECP gives one value per class, BVA adds the edges.
BVA with several variables: the GATE formulas
For a function of n independent variables, classical BVA uses the single fault assumption: failures are rarely caused by two variables being at extreme values at the same time. So we hold all variables at their nominal (middle) values and let one variable at a time take its boundary values.
| Variant | Values per variable | Number of test cases | n = 2 | n = 3 |
|---|---|---|---|---|
| Normal BVA | min, min+1, nom, max−1, max | 4n + 1 | 9 | 13 |
| Robustness testing | adds min−1, max+1 | 6n + 1 | 13 | 19 |
| Worst-case testing | 5 values, all combinations | 5n | 25 | 125 |
| Robust worst-case | 7 values, all combinations | 7n | 49 | 343 |
Why 4n + 1? Each variable contributes 4 non-nominal values (min, min+1, max−1, max) while the others stay nominal: 4n tests. The case where every variable is nominal is shared by all, so it is counted once: +1.
Example: a function emi(amount, months) with amount in [1, 100] (thousand rupees) and months in [1, 12]. Taking nominal values 50 and 6, normal BVA gives these 9 tests:
(amount, months)
(1, 6) (2, 6) (99, 6) (100, 6) ← amount varies, months nominal
(50, 1) (50, 2) (50, 11) (50, 12) ← months varies, amount nominal
(50, 6) ← all nominal (counted once)
Total = 4 × 2 + 1 = 9
Worst-case testing drops the single fault assumption and is used when variables interact strongly, or for safety-critical code; the cost grows exponentially, so it is used sparingly.
Decision Table Testing
ECP and BVA treat inputs one at a time. But many business rules depend on combinations of conditions. A decision table lists all combinations of conditions and the action the system must take for each.
A decision table has four parts: conditions (top left), condition entries (top right), actions (bottom left) and action entries (bottom right). Each column is a rule, and each rule becomes (at least) one test case.
Worked example: online shop offers
Rules from the specification: Delivery is free if the customer is a premium member or the order value is at least ₹500. A 10% discount is given only if the customer is a premium member and enters a valid coupon.
With 3 boolean conditions there are 23 = 8 rules:
| R1 | R2 | R3 | R4 | R5 | R6 | R7 | R8 | |
|---|---|---|---|---|---|---|---|---|
| C1: Premium member? | T | T | T | T | F | F | F | F |
| C2: Order ≥ ₹500? | T | T | F | F | T | T | F | F |
| C3: Valid coupon? | T | F | T | F | T | F | T | F |
| A1: Free delivery | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ||
| A2: 10% discount | ✔ | ✔ |
Collapsing the table. When two rules differ in only one condition and have identical actions, that condition does not matter there, and we can merge them using "–" (don't care):
R1 and R3 (member, coupon) give the same actions whatever the order value → "T, –, T".
R2 and R4 (member, no coupon) → "T, –, F".
R5 and R6 (not member, order ≥ ₹500) → "F, T, –".
R7 and R8 (not member, small order) → "F, F, –".
| R1' | R2' | R3' | R4' | |
|---|---|---|---|---|
| C1: Premium member? | T | T | F | F |
| C2: Order ≥ ₹500? | – | – | T | F |
| C3: Valid coupon? | T | F | – | – |
| A1: Free delivery | ✔ | ✔ | ✔ | |
| A2: 10% discount | ✔ |
The collapsed table needs 4 test cases instead of 8. A careful tester may still run the full 8 if the implementation might wrongly depend on the "don't care" condition (for example, a bug giving the discount to non-members who enter a coupon would only appear in R5 or R7).
Decision tables are also excellent for reviewing requirements: while filling the table you quickly spot combinations the specification forgot to mention, or contradictory rules.
Cause–effect graphing
For larger rule sets, cause–effect graphing (introduced by Myers) is used to build the decision table systematically. Causes (input conditions) and effects (outputs) are drawn as nodes and linked with Boolean operators AND, OR and NOT. Constraints between causes are marked, for example E (exclusive: at most one can be true), I (inclusive: at least one), O (one and only one) and R (requires). The graph is then converted into a decision table, and each column becomes a test case.
Causes Effects
C1 Premium member ─────┬──── OR ──────► E1 Free delivery
C2 Order ≥ ₹500 ──────┘
C1 Premium member ─────┬──── AND ─────► E2 10% discount
C3 Valid coupon ──────┘
State Transition Testing
Some systems respond differently to the same input depending on what happened before. An ATM accepts a PIN on the first and second attempt, but after the third wrong PIN it keeps the card. Such behaviour is described by a state transition diagram (you met state machine diagrams in UML Diagrams).
State transition testing designs test cases to exercise the states, events and transitions of a state model, including checking that invalid transitions are rejected.
Worked example: ATM PIN entry
wrong PIN wrong PIN wrong PIN
┌────────┐ ───────► ┌────────┐ ───────► ┌────────┐ ───────► ┌───────────────┐
│ S1 │ │ S2 │ │ S3 │ │ S5 │
│1st try │ │2nd try │ │3rd try │ │ Card retained │
└────────┘ └────────┘ └────────┘ └───────────────┘
│ │ │
│ correct PIN │ correct PIN │ correct PIN
▼ ▼ ▼
┌──────────────────────────────────────────────┐
│ S4 Access granted │
└──────────────────────────────────────────────┘
The same model as a state table, which also shows the invalid (missing) transitions:
| Current state | Event: correct PIN | Event: wrong PIN |
|---|---|---|
| S1 (1st try) | S4 | S2 |
| S2 (2nd try) | S4 | S3 |
| S3 (3rd try) | S4 | S5 |
| S4 (Access granted) | — (invalid) | — (invalid) |
| S5 (Card retained) | — (invalid) | — (invalid) |
Test cases:
All states coverage: visit S1…S5. Two tests are enough: "correct" (S1→S4) and "wrong, wrong, wrong" (S1→S2→S3→S5).
All valid transitions (also called 0-switch coverage): the 6 arrows. For example: T1 = correct; T2 = wrong, correct; T3 = wrong, wrong, correct; T4 = wrong, wrong, wrong. Together these cover all 6 transitions.
Invalid transitions: try to enter a PIN when the card has been retained (S5), and check that the system ignores it rather than granting access.
State transition testing is widely used for embedded systems, login and session handling, order workflows ("placed → paid → shipped → delivered") and protocol implementations.
Other Black-Box Techniques
Pairwise (all-pairs) testing
A web app must work on 2 operating systems (Android, iOS), 3 browsers (Chrome, Firefox, Edge) and 3 languages (English, Hindi, Tamil). Testing every combination needs 2 × 3 × 3 = 18 runs. Studies of real failures show that most configuration defects are triggered by a single parameter or by an interaction of just two parameters. Pairwise testing therefore only guarantees that every pair of values appears in at least one test:
| # | OS | Browser | Language |
|---|---|---|---|
| 1 | Android | Chrome | English |
| 2 | Android | Firefox | Hindi |
| 3 | Android | Edge | Tamil |
| 4 | iOS | Chrome | Hindi |
| 5 | iOS | Firefox | Tamil |
| 6 | iOS | Edge | English |
| 7 | Android | Chrome | Tamil |
| 8 | iOS | Firefox | English |
| 9 | Android | Edge | Hindi |
These 9 tests cover all 21 value pairs (6 OS–browser, 6 OS–language, 9 browser–language). Nine is the minimum here, because the 9 browser–language pairs alone need 9 different tests. The saving grows dramatically with more parameters; tools such as Microsoft's PICT generate such sets automatically.
Use case testing
Test cases are derived from use cases (see UML Diagrams): one test for the main success scenario and one for each alternative and exception flow. For "Withdraw cash", that means a normal withdrawal, insufficient balance, daily limit exceeded, ATM out of cash, and so on. Use case tests are natural for system and acceptance testing (see Levels of Testing).
Experience-based techniques
Error guessing: the tester uses experience to guess likely mistakes: empty input, zero, negative numbers, very long strings, special characters such as
'or<script>, leap days, duplicate submissions.Exploratory testing: simultaneous learning, test design and execution, usually in time-boxed sessions guided by a "charter" (for example, "explore fee payment with interrupted network").
Checklist-based testing: working through a list of conditions to check, built from past defects and standards.
Black-Box vs White-Box vs Grey-Box
| Aspect | Black-box | White-box | Grey-box |
|---|---|---|---|
| Basis for test design | Requirements / specification | Internal code structure | Specification plus partial knowledge of internals (e.g. database schema, architecture) |
| Knowledge of code needed | No | Yes | Partial |
| Typical techniques | ECP, BVA, decision tables, state transitions, use cases | Statement, branch, condition, path coverage | Matrix testing, regression based on known changes |
| Finds | Missing features, wrong behaviour, interface errors | Untested code, logic errors, dead code | Integration and data-flow problems |
| Main levels used | System, acceptance (also unit) | Unit, integration | Integration, system |
| Can start | As soon as requirements exist | Once code (or detailed design) exists | Once design is known |
Common Mistakes
Forgetting invalid classes. Testing only valid data checks the "happy path" and misses crashes and wrong error handling.
Putting several invalid values in one test. One error message can hide the fact that the second check is missing.
Testing only nominal values and skipping boundaries, as our example showed. ECP alone missed the
>vs>=fault.Counting 4n + 1 wrongly. Writing 5n counts the all-nominal test n times. It appears only once.
Applying BVA to unordered data. Boundaries only make sense for ordered values (numbers, dates, lengths). For a set like {MH, KA, TN}, use ECP.
Collapsing a decision table without checking the actions. Rules can only be merged when their actions are identical.
Thinking black-box testing means "testing without thinking". It requires careful analysis of the specification; it simply does not use the code.
Interview and Exam Questions
Q1. A function takes three independent integer inputs, each with a defined range. How many test cases are needed for normal BVA, robustness testing and worst-case testing?
Normal BVA: 4n + 1 = 4 × 3 + 1 = 13. Robustness: 6n + 1 = 19. Worst-case: 53 = 125. (Robust worst-case: 73 = 343.)
Q2. An input field accepts a username of 6 to 15 characters. Give the equivalence classes and the boundary values.
Classes: length < 6 (invalid), 6–15 (valid), length > 15 (invalid). Boundary values for length (3-value style): 5, 6, 7 and 14, 15, 16. Also consider length 0 (empty input) as a special invalid case worth testing.
Q3. Why do we need BVA if ECP already covers every class?
ECP assumes all values in a class behave the same, but the most common coding faults (wrong relational operator, off-by-one loop limits) affect only values at the edge of a class. A middle value such as 67 cannot reveal that > 60 was written instead of >= 60; the boundary value 60 can.
Q4. What is the "single fault assumption" in BVA?
It assumes failures are rarely caused by two or more variables taking extreme values at the same time. So BVA varies one variable at a time to its boundary values while keeping the others at nominal values, giving 4n + 1 tests instead of 5n.
Q5. A decision table has 4 independent boolean conditions. How many rules does the full table have, and when can it be reduced?
24 = 16 rules. It can be reduced by merging rules that differ in only one condition and have exactly the same actions; that condition becomes a "don't care" (–) entry.
Q6. What is the difference between 0-switch and 1-switch coverage in state transition testing?
0-switch coverage exercises every single valid transition at least once. 1-switch coverage exercises every valid sequence of two consecutive transitions, which can reveal faults that depend on the previous transition.
Q7. You have 4 parameters with 3 values each. How many tests does exhaustive testing need, and what is the minimum for pairwise testing?
Exhaustive: 34 = 81. Pairwise needs at least 3 × 3 = 9 (every pair of values of any two parameters must appear), and 9 is actually achievable here using an orthogonal array L9(34), which covers all pairs in exactly 9 tests.
Summary
Black-box testing designs tests from the specification only. It finds missing features, wrong outputs and interface errors, but cannot show which code was never executed.
Equivalence partitioning splits inputs into valid and invalid classes and tests one representative per class. Weak testing uses each class once; strong testing uses all combinations.
Boundary value analysis tests at and next to class edges, where off-by-one faults live. For n variables: normal 4n + 1, robust 6n + 1, worst-case 5n, robust worst-case 7n.
Decision tables (and cause–effect graphs) handle combinations of conditions; n boolean conditions give 2n rules, which can be collapsed with "don't care" entries.
State transition testing covers states, valid transitions and invalid transitions of event-driven behaviour.
Pairwise testing, use case testing and experience-based techniques (error guessing, exploratory testing) complete the toolkit.
Next, White-Box Testing & Cyclomatic Complexity opens the box and measures how thoroughly tests exercise the code itself.