Everyone wants "high-quality software", but ask five people what that means and you will get five answers. A student says "no bugs". A user says "easy to use and fast". A manager says "delivered on time". A maintenance engineer says "code I can understand and change". All of them are right, and that is the problem: quality has many dimensions, and you cannot improve what you cannot define or measure.
This chapter defines software quality, presents the classic and current quality models (McCall, Boehm, ISO/IEC 25010), explains quality assurance and process standards (ISO 9001, CMMI, Six Sigma), and then covers the product metrics most asked in exams: Halstead's software science, defect density, defect removal efficiency and the object-oriented CK metrics. Size metrics (LOC and function points) were covered in Software Size Estimation: LOC & Function Points, and cyclomatic complexity in White-Box Testing & Cyclomatic Complexity.
Software quality is the degree to which a software product satisfies stated and implied needs when used under specified conditions. In simpler words: conformance to explicitly stated functional and non-functional requirements, to documented development standards, and to the implicit characteristics expected of professionally developed software.
Why Quality Is Hard to Pin Down
Quality has two sides:
Quality of design (fitness for purpose): do the requirements and design describe what users really need?
Quality of conformance: does the implementation actually follow that design and those requirements?
A product can conform perfectly to a poor specification and still be useless (the "absence-of-defects fallacy" from Software Testing Fundamentals). It can also aim at the right needs but be full of defects. Good quality needs both.
Some quality attributes are external, visible to users while the software runs (reliability, usability, performance), while others are internal, visible only to developers (modularity, complexity, readability). Internal attributes can be measured early, from code and design, and are useful because they predict external ones: highly complex, tightly coupled code tends to be less reliable and harder to maintain.
Quality Models
A quality model breaks the vague idea of "quality" into a hierarchy of named characteristics, each of which can be measured. Three are important.
McCall's quality model (1977)
Developed for the US Air Force, McCall's model groups 11 quality factors under three perspectives on a software product:
McCall's quality triangle
Product REVISION Product TRANSITION
(can I change it?) (can I adapt it to new environments?)
• Maintainability • Portability
• Flexibility • Reusability
• Testability • Interoperability
╲ ╱
╲ ╱
╲ ╱
Product OPERATION
(how well does it run?)
• Correctness • Reliability • Efficiency
• Integrity • Usability
| Factor | Meaning |
|---|---|
| Correctness | Satisfies its specification and the user's objectives |
| Reliability | Performs its intended function with required precision, without failure |
| Efficiency | Uses computing resources (CPU, memory) economically |
| Integrity | Controls access by unauthorised persons (security) |
| Usability | Effort to learn, operate, prepare input and interpret output |
| Maintainability | Effort to locate and fix an error |
| Flexibility | Effort to modify the operational program |
| Testability | Effort to test the program to ensure it performs its function |
| Portability | Effort to transfer it to another hardware/software environment |
| Reusability | Extent to which parts can be reused in other applications |
| Interoperability | Effort to couple it with another system |
Each factor is further linked to measurable criteria (such as simplicity, modularity, self-descriptiveness, traceability) which are scored with metrics. This factor → criteria → metrics hierarchy is McCall's lasting contribution.
Boehm's quality model (1978)
Boehm's model is also hierarchical. At the top it asks what the user wants from the software: as-is utility (how well can I use it as it is: reliability, efficiency, human engineering), maintainability (how easy is it to understand, modify and test) and portability (can I use it in a new environment). These break down into intermediate characteristics and then into primitive measurable characteristics (device independence, self-containedness, accuracy, completeness and so on). Compared with McCall it gives more weight to maintainability and to the program's understandability.
ISO/IEC 25010:2023 (the current international standard)
ISO/IEC 25010 belongs to the SQuaRE family (Systems and software Quality Requirements and Evaluation, ISO/IEC 25000 series). The 2011 edition had 8 characteristics; the current 2023 edition has 9 product quality characteristics:
| Characteristic | Subcharacteristics |
|---|---|
| Functional suitability | Functional completeness, correctness, appropriateness |
| Performance efficiency | Time behaviour, resource utilisation, capacity |
| Compatibility | Co-existence, interoperability |
| Interaction capability (was "usability") | Appropriateness recognisability, learnability, operability, user error protection, user engagement, inclusivity, user assistance, self-descriptiveness |
| Reliability | Faultlessness (was "maturity"), availability, fault tolerance, recoverability |
| Security | Confidentiality, integrity, non-repudiation, accountability, authenticity, resistance |
| Maintainability | Modularity, reusability, analysability, modifiability, testability |
| Flexibility (was "portability") | Adaptability, scalability, installability, replaceability |
| Safety (new in 2023) | Operational constraint, risk identification, fail safe, hazard warning, safe integration |
Many university syllabi still teach the 2011 version (8 characteristics, with "usability" and "portability"); if your exam asks for ISO 9126 or 25010:2011, answer with those names, but know that the 2023 edition is current. (ISO/IEC 9126, the older standard with 6 characteristics, was replaced by 25010 in 2011.)
Comparing the models
| Aspect | McCall (1977) | Boehm (1978) | ISO/IEC 25010:2023 |
|---|---|---|---|
| Structure | 3 perspectives → 11 factors → criteria → metrics | 3 high-level uses → intermediate → primitive characteristics | 9 characteristics → subcharacteristics |
| Strength | First systematic model; links factors to metrics | Emphasis on maintainability and understandability | International consensus; includes security and safety |
| Status | Historical, exam favourite | Historical | Current standard |
Quality Assurance and Quality Control
| Aspect | Quality Assurance (QA) | Quality Control (QC) |
|---|---|---|
| Focus | Process: prevent defects | Product: detect defects |
| Nature | Proactive, planned, organisation-wide | Reactive, on specific work products |
| Activities | Defining standards and processes, audits, training, process improvement | Testing, reviews and inspections of work products |
| Question | "Are we following a good process?" | "Does this product meet the requirements?" |
Software Quality Assurance (SQA) activities include preparing an SQA plan, participating in defining the process, reviewing activities and work products for compliance, ensuring deviations are documented and handled, and reporting non-compliance to management.
Reviews: walkthroughs and inspections
Reviews are static techniques (no code is run) and are among the most cost-effective ways to find defects, because they find them early.
Walkthrough: informal; the author leads the team through the document or code, explaining it, and participants ask questions and raise issues. Main aim: understanding and knowledge sharing, plus finding defects.
Technical review: peers evaluate a work product for technical suitability, often with a moderator.
Inspection (Fagan inspection, 1976): the most formal: defined roles (moderator, author, reader, recorder, inspectors), entry and exit criteria, checklists, individual preparation, a logging meeting, rework and follow-up, and collection of metrics. Main aim: find as many defects as possible.
Cost of quality
The cost of quality is commonly divided into:
Prevention costs: training, process definition, quality planning.
Appraisal costs: reviews, testing, test tools.
Internal failure costs: rework and retesting for defects found before release.
External failure costs: defects found after release: support calls, patches, refunds, lost reputation, possibly legal costs.
Money spent on prevention and appraisal is usually far smaller than the external failure costs it avoids.
Process Quality Standards
ISO 9001
ISO 9001 specifies requirements for a quality management system in any kind of organisation, not just software. An organisation is certified by an external auditor. It requires documented processes, customer focus, management responsibility, internal audits, corrective actions and continual improvement (the Plan–Do–Check–Act cycle). The current edition is ISO 9001:2026 (published in September 2026, replacing ISO 9001:2015); organisations certified to the 2015 edition have a three-year transition period to move to it. ISO/IEC/IEEE 90003 gives guidance on applying ISO 9001 to software.
CMMI
The Capability Maturity Model Integration (originally from the Software Engineering Institute at Carnegie Mellon University, now maintained by ISACA's CMMI Institute; the current version is CMMI V3.0, released in 2023) rates how mature an organisation's processes are. It grew out of the earlier CMM for software. Its five maturity levels:
| Level | Name | Characteristics |
|---|---|---|
| 1 | Initial | Ad hoc, unpredictable; success depends on individual heroics |
| 2 | Managed | Projects are planned, monitored and controlled; requirements and configurations managed |
| 3 | Defined | Organisation-wide standard processes, tailored for each project |
| 4 | Quantitatively Managed | Processes measured and controlled using statistical and quantitative techniques |
| 5 | Optimizing | Continuous process improvement based on quantitative understanding and innovation |
A useful memory hook: Initial → Managed → Defined → Quantitatively managed → Optimizing ("I May Do Quality Overtime"). Many large Indian IT services companies have been appraised at maturity level 5.
Six Sigma
Six Sigma is a data-driven method for reducing variation and defects, originally from Motorola. A "six sigma" process produces at most 3.4 defects per million opportunities. Existing processes are improved with DMAIC (Define, Measure, Analyse, Improve, Control); new processes are designed with DMADV (Define, Measure, Analyse, Design, Verify).
| Aspect | ISO 9001 | CMMI | Six Sigma |
|---|---|---|---|
| Type | Certification standard (pass/fail) | Maturity model (levels 1–5) | Improvement methodology |
| Scope | Any industry | Developing products and services, especially software | Any process |
| Focus | Having a documented, working quality system | Progressive process maturity | Statistical defect reduction |
Software Metrics
A measure is a quantitative value of some attribute (e.g. 25 defects). A metric relates measures to give insight (e.g. 1.2 defects per KLOC). An indicator is a metric or combination of metrics that gives insight into a process, project or product, so that decisions can be made.
As introduced in Software Project Management, metrics fall into process, project and product metrics. Here we focus on product and quality metrics.
Halstead's software science
Maurice Halstead (1977) treated a program as a collection of operators and operands and derived several metrics from four basic counts:
n1 = number of distinct operators, n2 = number of distinct operands
N1 = total occurrences of operators, N2 = total occurrences of operands
| Metric | Formula |
|---|---|
| Program vocabulary | n = n1 + n2 |
| Program length | N = N1 + N2 |
| Estimated length | N̂ = n1 log2 n1 + n2 log2 n2 |
| Volume (bits) | V = N × log2 n |
| Difficulty | D = (n1 / 2) × (N2 / n2) |
| Program level | L = 1 / D |
| Effort | E = D × V |
| Time to program (seconds) | T = E / 18 |
| Estimated delivered bugs | B = V / 3000 |
The constant 18 (the "Stroud number") is the assumed number of elementary mental discriminations per second; 3000 is an empirical constant for bugs.
Worked example
sum = 0;
for (i = 1; i <= n; i++)
sum = sum + i;
Counting conventions vary between textbooks, so always state yours. Here we treat for ( ) as one operator, count each ; that ends a statement or clause, and count variables and constants as operands.
| Operator | Count | Operand | Count | |
|---|---|---|---|---|
= | 3 | sum | 3 | |
; | 4 | i | 4 | |
for ( ) | 1 | n | 1 | |
<= | 1 | 0 | 1 | |
++ | 1 | 1 | 1 | |
+ | 1 | |||
| n1 = 6 | N1 = 11 | n2 = 5 | N2 = 10 |
Now the metrics:
Vocabulary n = 6 + 5 = 11
Length N = 11 + 10 = 21
Estimated length N̂ = 6 log2 6 + 5 log2 5 = 15.51 + 11.61 = 27.12
Volume V = 21 × log2 11 = 21 × 3.4594 = 72.65 bits
Difficulty D = (6 / 2) × (10 / 5) = 6
Level L = 1 / 6 = 0.167
Effort E = 6 × 72.65 = 435.9
Time T = 435.9 / 18 = 24.2 seconds
Bugs B = 72.65 / 3000 = 0.024
Halstead metrics are easy to compute automatically and correlate with size, but they ignore control structure and data structures, and the psychological assumptions behind T and B are weak. They are best treated as rough indicators.
Defect metrics
Defect density = number of defects ÷ size (KLOC or function points). Example: 30 defects in a 25 KLOC system → 30 / 25 = 1.2 defects per KLOC.
Defect removal efficiency (DRE) = E / (E + D), where E = errors/defects found before delivery and D = defects found after delivery. Example: 120 defects found during reviews and testing and 30 reported by users in the first months → DRE = 120 / 150 = 0.80 (80%). The ideal is 1. DRE can also be computed per phase: defects found in a phase ÷ (those + defects from that phase that escaped to later phases).
Mean time to failure and related reliability metrics are covered in Software Reliability.
Customer problems per user-month and similar measures track quality as users experience it.
Complexity and structure metrics
Cyclomatic complexity V(G) = E − N + 2P (see White-Box Testing & Cyclomatic Complexity).
Fan-in / fan-out (see Modularity, Coupling & Cohesion). Henry and Kafura combined them into an information-flow complexity: length × (fan-in × fan-out)2.
Object-oriented metrics: the CK suite
Chidamber and Kemerer (1994) proposed six metrics for object-oriented designs:
| Metric | Meaning | High value suggests |
|---|---|---|
| WMC Weighted Methods per Class | Sum of complexities of a class's methods (with all weights 1, simply the number of methods) | Class is complex, harder to maintain and reuse |
| DIT Depth of Inheritance Tree | Length of the path from the class to the root of the hierarchy | More inherited behaviour, harder to predict; but more reuse |
| NOC Number of Children | Number of immediate subclasses | More reuse, but more classes affected by changes; may indicate misuse of inheritance |
| CBO Coupling Between Objects | Number of other classes a class is coupled to | Harder to test, change and reuse |
| RFC Response For a Class | Number of methods that can run in response to a message (own methods + methods they directly call) | More testing and debugging effort |
| LCOM Lack of Cohesion in Methods | How much methods fail to share instance variables | Low cohesion; class may need splitting |
Example: in the hierarchy Vehicle → {Car, Bike}, Car → ElectricCar, DIT(Vehicle) = 0, DIT(ElectricCar) = 2, NOC(Vehicle) = 2 and NOC(Car) = 1.
Using metrics wisely
Metrics are powerful but easy to misuse. Goodhart's law warns that "when a measure becomes a target, it ceases to be a good measure": if developers are rewarded for low defect counts, defects stop being reported; if they are rewarded for LOC, code gets longer. The GQM (Goal–Question–Metric) approach of Basili helps: first state a goal ("improve the reliability of the fee module"), then the questions that tell whether it is met ("how many failures per week in production?"), and only then choose metrics that answer those questions.
Common Mistakes
Equating quality with "no bugs". Usability, performance, security, maintainability and more are all part of quality.
Mixing up QA and QC. QA is process-oriented and preventive; QC (including testing) is product-oriented and detective.
Listing McCall factors under the wrong perspective. Operation: correctness, reliability, efficiency, integrity, usability. Revision: maintainability, flexibility, testability. Transition: portability, reusability, interoperability.
Using total counts where distinct counts are needed in Halstead. Vocabulary uses n1, n2 (distinct); length uses N1, N2 (total). Difficulty uses n1, N2 and n2.
Writing DRE as D / (E + D). DRE is the fraction removed before delivery: E / (E + D).
Naming CMMI level 2 "Repeatable". That was the name in the old SW-CMM; in CMMI it is "Managed".
Interview and Exam Questions
Q1. A program has n1 = 10, n2 = 20, N1 = 60, N2 = 40. Find vocabulary, length, volume and difficulty.
n = 30, N = 100, V = 100 × log2 30 = 100 × 4.907 = 490.7, D = (10/2) × (40/20) = 10. Effort E = 10 × 490.7 = 4907.
Q2. During development 450 defects were removed. After release, users reported 50 defects. What is the DRE?
DRE = 450 / (450 + 50) = 450 / 500 = 0.90 (90%).
Q3. List McCall's three perspectives and one factor from each.
Product operation (e.g. reliability), product revision (e.g. maintainability), product transition (e.g. portability).
Q4. What changed in ISO/IEC 25010:2023 compared with the 2011 edition?
It has 9 characteristics instead of 8: safety was added; usability was renamed interaction capability and portability became flexibility (with scalability added); reliability's "maturity" became faultlessness; security gained resistance; and interaction capability gained subcharacteristics such as inclusivity, user engagement and self-descriptiveness.
Q5. What is the difference between a walkthrough and an inspection?
A walkthrough is informal and led by the author, mainly for understanding and finding issues. An inspection is formal, led by a trained moderator, with defined roles, preparation, checklists, entry/exit criteria, logged defects, rework and follow-up, and metrics collection.
Q6. At which CMMI maturity level are processes controlled using statistical techniques?
Level 4, Quantitatively Managed. Level 5 (Optimizing) then uses this quantitative understanding for continuous improvement.
Q7. A system of 40 KLOC had 60 defects found in testing. What is its defect density?
60 / 40 = 1.5 defects per KLOC (based on defects found in testing).
Summary
Software quality is conformance to explicit requirements, standards and implicit expectations; it has many dimensions, external and internal.
McCall: 11 factors under operation, revision and transition. Boehm: as-is utility, maintainability, portability. ISO/IEC 25010:2023: 9 characteristics, including the new safety and the renamed interaction capability and flexibility.
QA prevents defects through process; QC detects them in products. Reviews range from informal walkthroughs to formal Fagan inspections.
Process standards: ISO 9001 (certification), CMMI (maturity levels 1–5: Initial, Managed, Defined, Quantitatively Managed, Optimizing), Six Sigma (3.4 defects per million opportunities).
Halstead: n = n1 + n2, N = N1 + N2, V = N log2 n, D = (n1/2)(N2/n2), E = D × V. DRE = E / (E + D). Defect density = defects / KLOC. CK metrics: WMC, DIT, NOC, CBO, RFC, LCOM.
Next, Software Reliability looks in depth at one of the most important quality attributes and how to measure and predict it.