Test Coverage Defined

In white box testing we examine the source code to derive test cases. There are a variety of techniques which vary in how thoroughly they exercise the code. A common term, "Test Coverage" refers to the degree of thoroughness or how well the suite of test cases "covers" or exercises the code.

Statement Coverage (a.k.a. "Block Coverage")

Definition: Every statement in the program is executed at least once in some test.

This is a minimal testing requirement. (You would probably feel skeptical if there are statements in the program that the test suite doesn't even cause to execute.)

Even though it's a minimal requirement, it is difficult to achieve with ad-hoc, pound on the keyboard-style test construction.

Even test suites that produce 100% statement coverage will MISS certain defects relating to control structures. For example,

int hour;
...
String message = null;
if (hour < 12)
{
    message = "good morning ";
}
message = message.substring(5) + "world";

Branch Coverage (a.k.a. decision coverage)

definition: For every decision point in the code each branch is chosen at least once in some test. The entire boolean expression is considered one "decision point" regardless of whether it contains logical operators (AND, OR).

Branch coverage includes statement coverage since exercising every branch must lead to exercising every statement.  Branch coverage does not require exercising each simple condition in a compound condition.

A shortcoming of this measure is that it ignores the effect of short-circuit operators in evaluating compound conditions.  That is, you can cover all the branches without covering all the simple conditions.

Challenge: Create an example of a simple defective program and write a successful test case that demonstrates complete branch coverage. Now create a second test case that reveals the code is defective. View one solution.

Branch-Condition Coverage

That is, every statement and both alternatives for every simple condition will have been executed.

Branch-Condition coverage includes branch coverage (for all practical purposes), but also covers compound conditions.

Path Coverage

definition: Every distinct entry-exit path through the code is executed at least once in some test.

Path coverage requires considering every combination of every branch through the code, including implicit paths through every simple condition.

Loop Coverage

definition: Every iteration of every loop is executed at least once in some test. This is practically impossible.

See these definitions illustrated by example.

For more details and a discussion of other coverage styles, see this reference.

Coverage Tools

Exercise

READ temp
READ hour
IF hour > 12 OR temp < 70 THEN
   PRINT "eat lunch"
END IF
PRINT "go "
IF temp > 75 THEN
   PRINT "swim"
END IF