In Software Quality & Metrics we saw that reliability is one of the main characteristics in every quality model, from McCall to ISO/IEC 25010. It is also the one users notice first. Nobody praises a banking app for low coupling, but everyone remembers the day it crashed during a payment.
This chapter makes reliability measurable. We define it precisely, see why software fails differently from hardware, learn the standard metrics (MTTF, MTTR, MTBF, availability, ROCOF, POFOD), compute them from a real failure log, work through the exponential model and series/parallel systems, and then study reliability growth models such as Musa's model, which are popular in university exams. We end with the techniques used to build reliable software: fault avoidance, fault removal and fault tolerance.
Software reliability is the probability that a program operates without failure for a specified period of time in a specified environment.
Three parts of this definition matter, and exam answers often lose marks by dropping them:
Probability: reliability is a number between 0 and 1, not a yes/no property.
Specified period: "0.98 reliable" means nothing without a time (or number of transactions). Reliability for 1 hour is higher than for 1 month.
Specified environment: the same program can be very reliable for one group of users and unreliable for another, because they use different features and inputs.
Faults, Failures and Why Reliability Is About Users
Recall from Software Testing Fundamentals: a human error (mistake) creates a fault (defect, bug) in the code, and when that faulty code runs under certain conditions the program shows a failure, a visible deviation from expected behaviour.
Reliability counts failures, not faults. This leads to a surprising but important result, reported by Edward Adams in a well-known 1984 IBM study: a large share of faults in a product cause failures only after hundreds or thousands of years of use. Removing those faults barely changes the reliability users experience, while removing the few faults in heavily used code changes it a lot.
Program with 100 faults
├── 10 faults in the login / payment path → hit by almost every user, every day
└── 90 faults in rarely used admin reports → hit perhaps once a year
Fixing the 10 faults: failures drop sharply → reliability ↑↑
Fixing 30 of the 90: users hardly notice → reliability ↑ (tiny)
That is why reliability depends on the operational profile: the set of operations users perform and how often each occurs. For a fee portal it might look like this:
| Operation | Probability of occurrence |
|---|---|
| View fee status | 0.55 |
| Pay fees online | 0.30 |
| Download receipt | 0.12 |
| Admin: generate defaulter report | 0.03 |
Statistical (operational-profile) testing picks test inputs with these same probabilities, so the failures found during testing are the ones users would actually meet, and the measured failure rate predicts field reliability.
Hardware vs Software Reliability
Hardware wears out; software does not. This difference changes the shape of the failure-rate curve.
HARDWARE: bathtub curve SOFTWARE: idealized vs actual
failure failure
rate │\ / rate │\ spikes after each change
│ \ / │ \ /\ /\ /\
│ \ / │ \ / \ / \ / \__/ actual
│ \______________/ │ \/ \_/ \_/ (rises)
│ infant useful wear-out │ \_______________________ idealized
│ mortality life │
└──────────────────────────► time └──────────────────────────► time
Hardware has infant mortality (manufacturing defects), a long flat useful life, and then wear-out as components physically degrade.
Software ideally has high failure rate early (undiscovered faults), which falls as faults are fixed and then stays flat: code does not rust.
In reality every change can introduce new faults, so the failure rate spikes after each modification and the minimum slowly creeps upward. Software "deteriorates" because of change, not wear. (This picture follows Pressman.)
| Aspect | Hardware | Software |
|---|---|---|
| Main cause of failure | Physical wear, manufacturing defects, environment (heat, dust) | Design and coding faults present from the start |
| Ageing | Wears out | Does not wear out; deteriorates through change |
| Repair | Replace the failed part with an identical one | Change the design/code; the "repair" alters the product |
| Redundancy with identical copies | Very effective (independent failures) | Useless: identical copies have identical faults |
| Failure after fix | Same part can fail again later | A fixed fault stays fixed, but the fix may add new faults |
Reliability Metrics
Different systems need different metrics. A telephone switch cares about how often it is down; an ATM cares about the chance that a single withdrawal fails. The common metrics are:
| Metric | Meaning | Good for |
|---|---|---|
| MTTF (Mean Time To Failure) | Average operating time between restart and the next failure | Systems with long transactions or continuous running |
| MTTR (Mean Time To Repair) | Average time to restore service after a failure | Measuring recovery speed |
| MTBF (Mean Time Between Failures) | MTTF + MTTR: average time from one failure to the next | Repairable systems |
| ROCOF (Rate Of Occurrence Of Failures) | Failures per unit operating time, e.g. 2 failures / 1000 hours | Systems with regular, frequent demands (e.g. booking system) |
| POFOD (Probability Of Failure On Demand) | Probability that one request fails, e.g. 0.001 = 1 in 1000 demands | Rarely used but critical functions (airbag, emergency shutdown) |
| Availability | Fraction of time the system is usable: MTTF / (MTTF + MTTR) | Continuously running services (web sites, servers) |
◄──────── MTBF ────────►
◄───── MTTF ─────►◄MTTR►
──┼─────────────────X~~~~~┼─────────────────X~~~~~┼──► time
up fail repaired fail repaired
MTBF = MTTF + MTTR
Availability = MTTF / (MTTF + MTTR) = MTTF / MTBF
Note on names: textbooks differ. Pressman and most exam questions use MTBF = MTTF + MTTR. In hardware datasheets "MTBF" is often used loosely for the mean operating time between failures (what we call MTTF). Always state which convention you use; when MTTR is tiny the two are almost equal.
Why availability alone is not enough
System A fails once a month and takes 7 hours to fix. System B fails every 10 minutes but restarts in 1 second. Both can show about 99% availability, yet B is useless for a 30-minute exam submission. Reliability (how long it runs without failing) and availability (what fraction of time it is up) answer different questions.
Worked example: metrics from a failure log
A college fee portal ran for a 30-day trial (720 hours). Operations staff recorded when each failure happened and how long it took to restore service. Let's compute the metrics with a short Python script.
# Failure log of a college fee portal during a 30-day (720-hour) trial.
# Each entry: (hour at which failure occurred, hours taken to restore service)
log = [(52, 2.0), (131, 1.0), (240, 3.0), (318, 0.5),
(430, 1.5), (515, 2.5), (601, 1.0), (690, 0.5)]
total_hours = 720
downtime = sum(r for _, r in log)
uptime = total_hours - downtime
n = len(log)
mttf = uptime / n # mean operating time per failure
mttr = downtime / n # mean repair time
mtbf = mttf + mttr
availability = mttf / (mttf + mttr)
rocof = n / uptime # failures per operating hour
print(f"Failures : {n}")
print(f"Total downtime : {downtime:.1f} h")
print(f"Total uptime : {uptime:.1f} h")
print(f"MTTF : {mttf:.2f} h")
print(f"MTTR : {mttr:.2f} h")
print(f"MTBF = MTTF + MTTR : {mtbf:.2f} h")
print(f"Availability : {availability:.4%}")
print(f"ROCOF : {rocof:.5f} failures/h (~{rocof*1000:.1f} per 1000 h)")
Output:
Failures : 8
Total downtime : 12.0 h
Total uptime : 708.0 h
MTTF : 88.50 h
MTTR : 1.50 h
MTBF = MTTF + MTTR : 90.00 h
Availability : 98.3333%
ROCOF : 0.01130 failures/h (~11.3 per 1000 h)
Step by step: total repair time = 2 + 1 + 3 + 0.5 + 1.5 + 2.5 + 1 + 0.5 = 12 h, so uptime = 720 − 12 = 708 h. MTTF = 708 / 8 = 88.5 h, MTTR = 12 / 8 = 1.5 h, MTBF = 90 h, availability = 88.5 / 90 ≈ 98.33%. (Total operating time divided by number of failures is the usual estimate of MTTF from field data.)
98.33% sounds high, but it means about 12 hours of downtime a month, which is poor for an online payment service. The next table shows why engineers talk in "nines".
The "nines" of availability
| Availability | Downtime per year | Downtime per 30 days |
|---|---|---|
| 99% ("two nines") | 87.6 hours | 7.2 hours |
| 99.9% ("three nines") | 8.76 hours | 43.2 minutes |
| 99.99% ("four nines") | 52.6 minutes | 4.32 minutes |
| 99.999% ("five nines") | 5.26 minutes | 26 seconds |
Modern cloud teams state this as a Service Level Objective (SLO), for example "99.9% of requests succeed over 30 days". The allowed 0.1% of failure is the error budget: while budget remains the team may ship risky changes; when it is used up, they slow down and focus on reliability. This is the Site Reliability Engineering (SRE) view of the same idea.
POFOD example
An emergency-stop function is tested with 1000 demands drawn from its operational profile and fails 3 times. POFOD ≈ 3 / 1000 = 0.003. If the requirement is POFOD ≤ 0.001, the function is not yet acceptable.
The Exponential Reliability Model
If failures occur at a constant failure rate λ (failures per hour), the probability of running for time t without failure is:
R(t) = e^(−λt) reliability: probability of no failure in [0, t]
F(t) = 1 − e^(−λt) probability of at least one failure by time t
MTTF = 1 / λ
The constant-rate assumption fits the flat part of the curves above: a stable program between changes, used under a fixed operational profile.
Worked example
A server application has MTTF = 500 hours, so λ = 1/500 = 0.002 failures/hour.
| Mission time t | λt | R(t) = e^(−λt) |
|---|---|---|
| 1 hour | 0.002 | 0.9980 |
| 8 hours (one working day) | 0.016 | 0.9841 |
| 100 hours | 0.2 | 0.8187 |
| 500 hours (= MTTF) | 1 | 0.3679 |
Look at the last row: the probability of surviving until the MTTF is only e^(−1) ≈ 36.8%. MTTF is an average, not a guarantee. Most runs fail before the mean time because a few very long runs pull the average up.
Reliability of Systems: Series and Parallel
Real systems are built from components (web server, application, database). Assuming components fail independently:
SERIES (all must work) PARALLEL (at least one must work)
──[R1]──[R2]──[R3]── ┌──[R1]──┐
──┤ ├──
R = R1 × R2 × R3 └──[R2]──┘
R = 1 − (1 − R1)(1 − R2)
Worked example
Series: front end 0.99, application 0.98, database 0.95. R = 0.99 × 0.98 × 0.95 = 0.9217. The system is less reliable than its weakest part.
Parallel: two application servers of 0.90 each. R = 1 − (0.10)(0.10) = 0.99. Three of them: 1 − 0.1³ = 0.999.
Mixed: two parallel app servers of 0.95 each, in series with a database of 0.99. Parallel part = 1 − 0.05² = 0.9975; whole system = 0.9975 × 0.99 = 0.9875 (0.987525).
Warning for software: the parallel formula assumes independent failures. Two copies of the same program fed the same input fail together, so duplicating software protects against hardware crashes but not against a software fault. That is the motivation for N-version programming, below.
Reliability Growth Models
During testing, each failure leads to a fix, so the failure rate should fall and reliability should grow. A reliability growth model fits a curve to the failure data observed so far and predicts: how reliable is the software now, and how much more testing is needed to reach the target?
Jelinski–Moranda model (1972)
One of the earliest models. Assumptions: the program starts with N faults, each fault contributes the same amount φ to the failure rate, and each fault is fixed perfectly as soon as it causes a failure. After (i − 1) faults are removed, the failure rate during the i-th interval is:
λᵢ = φ × (N − i + 1)
Example: N = 50, φ = 0.02 failures/hour per fault. After 20 fixes, 30 faults remain, so λ = 0.02 × 30 = 0.6 failures/hour and the expected time to the next failure is 1/0.6 ≈ 1.67 hours. Each fix reduces λ by the same step φ.
Musa's basic execution time model
John Musa's model (1975) measures time as execution (CPU) time τ, not calendar time, because faults are exposed only while the program actually runs. It assumes a finite total number of failures ν₀ that would occur in infinite time, and a failure intensity that falls linearly with the number of failures experienced μ:
λ(μ) = λ₀ (1 − μ / ν₀) failure intensity after μ failures
μ(τ) = ν₀ (1 − e^(−λ₀τ / ν₀)) expected failures by execution time τ
λ(τ) = λ₀ e^(−λ₀τ / ν₀) failure intensity at execution time τ
To go from present intensity λP to a target (final) intensity λF:
Δμ = (ν₀ / λ₀) (λP − λF) additional failures to experience
Δτ = (ν₀ / λ₀) ln(λP / λF) additional execution time needed
λ₀ = initial failure intensity, ν₀ = total expected failures
Worked example (classic exam style)
A program has initial failure intensity λ₀ = 10 failures/CPU-hour and is expected to experience ν₀ = 100 failures in total. So far 50 failures have occurred.
Current intensity: λP = 10 × (1 − 50/100) = 5 failures/CPU-hour.
Target λF = 2 failures/CPU-hour. Additional failures: Δμ = (100/10) × (5 − 2) = 30 failures.
Additional testing time: Δτ = (100/10) × ln(5/2) = 10 × 0.9163 = 9.16 CPU-hours.
After 5 CPU-hours of testing from the start: μ(5) = 100 × (1 − e^(−0.5)) ≈ 39.3 failures, and λ(5) = 10 × e^(−0.5) ≈ 6.07 failures/CPU-hour.
Notice the diminishing returns: halving the intensity from 10 to 5 needed 50 failures, while going from 5 to 2 needs only 30 more failures but each costs more test time, because failures are now rarer.
Other models you should recognise
| Model | Key idea |
|---|---|
| Musa–Okumoto logarithmic Poisson | λ(μ) = λ₀ e^(−θμ): early fixes reduce intensity more than later ones; total failures are unbounded |
| Goel–Okumoto (NHPP) | Expected failures by time t: m(t) = a(1 − e^(−bt)), with a = total expected faults, b = detection rate |
| Littlewood–Verrall | Bayesian: treats failure rates as random, allowing for imperfect fixes |
| Basic "step function" view | Reliability improves by a fixed step with each fix (equal fault sizes) |
All of these share a weakness: they assume testing follows the operational profile and that fixes rarely introduce new faults. When those assumptions break, predictions can be badly wrong, so teams compare several models against the same data.
Building Reliable Software
Measuring is only half the job. Reliability is achieved by four complementary strategies:
| Strategy | Goal | Examples |
|---|---|---|
| Fault avoidance (prevention) | Do not introduce faults | Clear requirements, good design, type-safe languages, coding standards, formal methods |
| Fault detection and removal | Find and fix faults before release | Reviews, inspections, static analysis, all the testing techniques of earlier chapters |
| Fault tolerance | Keep working even when a fault is triggered | Exception handling, retries, recovery blocks, N-version programming, checkpoints |
| Fault forecasting | Estimate current and future reliability | Reliability growth models, operational-profile testing |
Fault-tolerance techniques
N-version programming: N independent teams build N versions from the same specification; all run on the same input and a voter takes the majority output. Used in some avionics. Weakness: teams tend to make similar mistakes, especially where the specification is unclear, so failures are less independent than hoped.
Recovery blocks (Randell): run a primary algorithm, check its result with an acceptance test; if the test fails, restore the saved state and try an alternate algorithm. Versions run one after another, not in parallel.
Checkpointing and rollback: periodically save state; after a failure, roll back to the last checkpoint instead of restarting from scratch (databases do this with logs).
Graceful degradation: when a part fails, keep the core service running with reduced features (e.g. disable recommendations but keep checkout working).
N-VERSION PROGRAMMING RECOVERY BLOCK
input ─┬─► Version 1 ─┐ save state
├─► Version 2 ─┼─► VOTER ─► run Primary ──► acceptance test ── pass ──► output
└─► Version 3 ─┘ (majority) │ fail
restore state, run Alternate ──► test again ...
Reliability, Availability, Safety and Security
These four "dependability" attributes are often confused:
| Attribute | Question it answers | Example |
|---|---|---|
| Reliability | Will it run correctly for the required time? | Exam portal runs a 3-hour test without failing |
| Availability | Is it up when I need it? | Portal is reachable 99.9% of the month |
| Safety | Even if it fails, does it avoid harm to people or the environment? | Train control fails into "all signals red" |
| Security | Does it resist deliberate attacks? | Fees cannot be marked "paid" by tampering with a request |
A system can be safe but unreliable (it keeps shutting down safely) or reliable but unsafe (it works for years, then fails dangerously once).
Common Mistakes
Stating reliability without time and environment. "Reliability = 0.95" is incomplete; say "0.95 for 8 hours of operation under the student-usage profile".
Thinking MTTF is a guaranteed lifetime. Under the exponential model only about 37% of runs survive to the MTTF.
Mixing up MTTF and MTBF. MTBF includes repair time: MTBF = MTTF + MTTR.
Computing availability as MTTR / MTBF. That is the unavailability. Availability = MTTF / MTBF.
Counting faults instead of failures. Reliability is about failures seen by users; fault count alone does not predict it.
Using calendar time in Musa's basic model. It uses execution (CPU) time.
Assuming two copies of the same software are a parallel system. Identical software fails identically on the same input.
Interview and Exam Questions
Q1. Define software reliability. Why is it different from hardware reliability?
The probability of failure-free operation for a specified time in a specified environment. Hardware fails mainly through physical wear and follows a bathtub curve; software does not wear out but fails because of design faults, and its failure rate rises after changes that introduce new faults.
Q2. A system has MTTF = 380 hours and MTTR = 20 hours. Find MTBF and availability.
MTBF = 380 + 20 = 400 hours. Availability = 380 / 400 = 0.95 (95%).
Q3. With a constant failure rate of 0.002 per hour, what is the reliability for 100 hours?
R(100) = e^(−0.002 × 100) = e^(−0.2) ≈ 0.8187. MTTF = 1/0.002 = 500 hours.
Q4. Three modules with reliabilities 0.99, 0.98 and 0.95 work in series. What is the system reliability?
0.99 × 0.98 × 0.95 ≈ 0.9217.
Q5. In Musa's basic model, λ₀ = 10 failures/CPU-hr and ν₀ = 100. How many more failures and how much CPU time are needed to go from 5 to 2 failures/CPU-hr?
Δμ = (ν₀/λ₀)(λP − λF) = 10 × 3 = 30 failures. Δτ = (ν₀/λ₀) ln(λP/λF) = 10 × ln 2.5 ≈ 9.16 CPU-hours.
Q6. Which metric would you use for an aircraft's emergency landing-gear release software, and why?
POFOD, because the function is invoked rarely but must work when demanded. Time-based metrics like MTTF say little about a function that is idle almost all the time.
Q7. What is an operational profile and why does it matter?
A quantitative description of how users use the system: the operations and their probabilities. Testing according to it finds the failures users would see most often, and makes measured failure rates a valid predictor of field reliability.
Q8. Compare N-version programming and recovery blocks.
N-version runs several independently developed versions in parallel and votes on the output. Recovery blocks run a primary version, check it with an acceptance test and fall back to alternates sequentially after restoring state. N-version needs a voter and more hardware; recovery blocks need a good acceptance test. Both rely on design diversity.
Summary
Reliability = probability of failure-free operation for a specified time in a specified environment. It counts failures, not faults, and depends on the operational profile.
Software does not wear out; its failure rate spikes after changes. Identical copies give no protection against software faults.
MTBF = MTTF + MTTR, Availability = MTTF / (MTTF + MTTR). Other metrics: ROCOF (failures per unit time) and POFOD (probability a demand fails).
Constant failure rate: R(t) = e^(−λt), MTTF = 1/λ, and R(MTTF) ≈ 0.368.
Series: multiply reliabilities. Parallel: 1 − ∏(1 − Rᵢ), valid only for independent failures.
Musa basic model: λ(μ) = λ₀(1 − μ/ν₀); Δμ = (ν₀/λ₀)(λP − λF); Δτ = (ν₀/λ₀) ln(λP/λF).
Reliability is built by fault avoidance, removal, tolerance (N-version, recovery blocks, checkpoints) and forecasting.
Next, Software Maintenance & Reengineering looks at what happens after release, when most of the software's life and cost actually lie.