Black-box testing, boundary values, and object-oriented testing

Testing from the outside

Black-box testing (behavioural or functional testing) derives test cases from the specification, deliberately without reference to the internal structure. That deliberate blindness is a feature: it is the only way to find a function that was specified and never implemented, because there is no code path to cover for something that does not exist. Black-box techniques attack the input and output domains, the interfaces between components, and behaviour over time, and they complement rather than replace structural testing.

The first problem black-box testing has to solve is arithmetic. A field that accepts a duration in seconds admits millions of values and you can afford perhaps a dozen tests. The techniques below are two disciplined answers to which dozen.

Equivalence partitioning

Equivalence partitioning divides the input domain into classes whose members the specification says should all be treated identically. If that claim is true, testing one member of a class tests the class, and testing a second member adds no information. Classes come in valid and invalid flavours, and the invalid ones are usually where the defects are: for a numeric range you get below, inside and above; for a Boolean you get true and false; for a member of a set you get in the set and not in the set. The output of the technique is a small set of classes and one test case each — which is exactly the point at which most people stop, and exactly where the next technique earns its keep.

Boundary value analysis

Boundary value analysis rests on an empirical observation rather than a theory: defects cluster at the edges of the input and output domains rather than in their middles, because edges are where off-by-one errors, wrong comparison operators and inclusive/exclusive confusions live. Equivalence partitioning tells you which classes exist; BVA tells you where inside them to spend the tests. The rules are mechanical. For a range, test just below the low end, at the low end, at the high end, and just above the high end. For a count of n items, test with n − 1, n and n + 1. And — the half of the definition most answers omit — apply the same reasoning to the output domain: construct inputs that make the output land at its minimum, at its maximum, and one step beyond, because a fee cap or a display field has boundaries of its own.

Object-oriented testing: the class is the unit

One structural point changes when the software is object-oriented. The smallest testable unit is no longer the operation but the class, because an operation cannot be exercised meaningfully outside the state its object carries: the same method call behaves differently depending on what happened to the object before it. Class-level testing therefore drives sequences of operations against an instance. Two families are used. Random testing generates long, legal operation sequences and checks invariants after each one; partition testing groups operations by what they do to state — those that change it, those that only read it, and those that are driven by an external event — and tests within each group. Above the class, behavioural testing derives its sequences from the state diagram, exercising every transition at least once; for BorrowBox that means walking a tool through Available → Reserved → CheckedOut → Overdue → InInspection and back, including the transitions that are supposed to be illegal.

Worked example — the two-hour hold window

BorrowBox holds a reserved tool for two hours; the collection token is valid while the elapsed time since reservation is at most 02:00:00, and after that the hold lapses and the tool returns to the pool. Equivalence partitioning gives three classes: before the reservation exists (invalid), within the hold (valid), and after the hold has lapsed (valid input, different behaviour). One test each is three test cases and would very probably pass on broken code.

Boundary value analysis then places three more probes on the only live edge: 01:59:59, 02:00:00 and 02:00:01. Those three separate the four plausible implementations — <, <=, and the two versions that are one second out — and they are the tests that catch the developer who wrote elapsed < HOLD when the specification said at most. On the output side, the late-fee cap of forty dollars gets the same treatment: construct returns that produce 39.99, 40.00 and 40.01 before capping, and check the third comes back as 40.00. That last probe is a genuine output-domain boundary, and it is the one most candidates never mention.