Reviews: defect amplification, metrics, and choosing a review type

Error, defect, failure — say which convention you are using

Pressman uses a temporal convention, and it is worth stating deliberately because it is not the mainstream one. An error is a quality problem found by software engineers before the software is released — either to end users, or to the next framework activity. A defect (equivalently a fault) is a quality problem found after that release. A failure is the observed incorrect behaviour when a fault is executed under conditions that reach it. The wider community usually treats error, fault, defect and bug as synonyms and reserves a causal chain instead: a human mistake produces a fault in the artefact, which produces a failure when executed. Both conventions are defensible; using one without saying so is what loses marks. Name the convention in your first sentence and you are safe under either marker.

Defect amplification

The reason the distinction matters is economic. An error introduced during requirements modelling and not detected is carried forward into design, where it does not merely persist — it generates further errors, because the design is elaborating something wrong. Those errors, undetected, are carried into construction and generate more still. One early error becomes many late ones. That is defect amplification; defect propagation is the term for the downstream damage an undiscovered error does to later activities and to product behaviour. Reviews exist to interrupt the chain, and the amplification argument is the whole justification for their cost: you can pay now, in a two-hour meeting, or pay much more later, in rework across three artefacts. This is also exactly what accumulating technical debt looks like from the quality side.

Review metrics — proving the meeting paid for itself

A small set of measures is enough. Preparation effort (person-hours spent before the meeting), assessment effort (hours in it), rework effort (hours correcting what it found) — their sum is the review effort. Work product size, in whatever unit fits: UML models, document pages, lines of code. Errors found, split into minor and major by whether correcting one exceeds a threshold, summing to the total. From those two you get the workhorse:

Error density = total errors found / work product size.

Density is what makes a review predictable. Once you have averages across many reviews, a 40-page requirements model at a historical 0.68 errors per page should yield about 27 errors. Finding nine means one of two things — an unusually good document, or an inadequate review — and knowing which is a question worth asking. The second metric is defect removal efficiency, DRE = E / (E + D): errors found before release over all problems found before and after. DRE of 1 is the unreachable ideal, and a falling DRE is the earliest signal that a quality programme is decaying.

Four characteristics set the formality

How formal should a review be? Four characteristics, and formality rises as each is added: distinct roles explicitly assigned to reviewers; planning and preparation ahead of the meeting; a defined meeting structure with tasks and internal work products; and follow-up — verification that the corrections were actually made. A fifth consideration sits outside the model: frequency. A team on two-week sprints reviewing continuously can afford less ceremony per review, because defects are caught sooner and more often anyway. Match the formality to the criticality of the product, the schedule and the people — not to a house style.

Worked example — thirty BorrowBox errors, two histories

Suppose the BorrowBox increment introduces thirty errors in total: ten during requirements, eight during design, twelve during construction. Assume, conservatively, that each error carried into the next activity spawns one more, and that a run review catches 70% of what is in front of it. Without reviews: ten requirements errors reach design, amplify to twenty, plus eight new — twenty-eight reach construction, amplify to fifty-six, plus twelve new, so sixty-eight errors arrive at testing. With reviews: three survive the requirements review, amplify to six, plus eight new is fourteen, of which the design review leaves four; those amplify to eight, plus twelve new is twenty, of which the code review leaves six. Sixty-eight versus six, from the same thirty errors. The three review meetings cost perhaps twelve person-hours; the sixty-two errors they kept out of testing would not.