Test Coverage by Example
a.
|
b.
|
|
|
|
y = 10;
if (x > 1)
{
y = x / 0;
}
z = x + y;
|
Test Suite 1
|
Input
|
Expected
|
Pass?
|
a.
|
x=0
|
z=10
|
|
b.
|
x = -1
|
z=9 |
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
All our tests pass but we have only ____% statement coverage.
The statements not exercised might be defective.
How do we improve coverage? Invent more test data that forces
all the statements to be executed.
What if x = 5?
a.
|
|
|
y = 0;
if (x > 1)
{
y = x + 5;
}
z = x / y; |
Test Suite 2
|
Input
|
Expected
|
Pass?
|
a.
|
x=5
|
z= 0.5
|
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
All our tests pass and we have 100% statement coverage! Are we
done?
But what if x = 0?
a.
|
|
|
y = 0;
if (x > 1)
{
y = x + 5;
}
z = x + y; |
Test Suite 3
|
Input
|
Expected
|
Pass?
|
Branch (T or F)?
|
a.
|
x=5
|
z= 15
|
|
|
b.
|
x=0
|
z=0
|
|
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
Number of branches executed by the suite (b): ______
Total number of branches(n): _______
% branch coverage (b/n): ________
So branch coverage is better than statement coverage.
a.
|
|
|
s = 3;
if (x > 1 || y == 0 )
{
s = x / y;
}
z = s * 2;
|
Test Suite 4
|
Input
|
Expected
|
Pass?
|
Branch (T or F)? |
a.
|
x=2, y=2
|
z=2
|
|
|
b.
|
x=0, y=2
|
z=6;
|
|
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
Number of branches executed by the suite (b): ______
Total number of branches(n): _______
% branch coverage (b/n): ________
All our tests pass and we have 100% branch coverage! Are we
done?
But what if y = 0?
a.
|
|
|
s = 3;
if (x > 1 || y == 0 )
{
s = x - y;
}
z = s * 2;
|
Test Suite 5
|
Input
|
Expected
|
Pass?
|
Condition1 (T or F)? |
Condition2 (T or F)?
|
a.
|
x=2, y=2
|
z=0
|
|
|
|
b.
|
x=0, y=2
|
z=6
|
|
|
|
c.
|
x=-2, y=0
|
z= -4
|
|
|
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
Number of branches executed by the suite (b): ______
Total number of branches(n): _______
% branch coverage (b/n): ________
Number of branch-conditions executed by the suite (c): ______
Total number of branch-conditions(m): _______
% branch-condition coverage (c/m): ________
So branch-condition coverage is better than branch coverage.
a.
|
|
|
d = 2;
if ( x > 0 )
{
m = x - d;
d = 0;
}
else
{
m = x + d;
}
if ( y > 2 )
{
n = y - d;
}
else
{
n = y / d;
}
z = m + n;
|
Test Suite 6
|
Input
|
Expected
|
Pass?
|
Branch1? |
Branch2?
|
a.
|
x=3, y=3
|
z=4
|
|
|
|
b.
|
x=0, y=1
|
z=2.5
|
|
|
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
Number of branches executed by the suite (b): ______
Total number of branches(n): _______
% branch coverage (b/n): ________
All our tests pass and we have
100% branch coverage! There are no complex conditionals. Are we done?
But what if x=3, y = 1?
a.
|
|
|
d = 2;
if ( x > 0 )
{
m = x - d;
d = 0;
}
else
{
m = x + d;
}
if ( y > 2 )
{
n = y - d;
}
else
{
n = y + d;
}
z = m + n;
|
Test Suite 7
|
Input
|
Expected Pass?
|
Branch1? |
Branch2?
| Path #
|
a.
|
x=3, y=3
|
z=4
|
|
|
___
|
b.
|
x=0, y=1
|
z=5
|
|
|
___
|
c.
|
x=0, y=3
|
z=3
|
|
|
___
|
d.
|
x=3, y=1
|
z=2
|
|
|
___
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
Number of branches executed by the suite (b): ______
Total number of branches(n): _______
% branch coverage (b/n): ________
Number of paths executed by the suite (p): ______
Total number of paths (q): _______
% path coverage (p/q): ________
So path coverage is better than branch coverage.
Here's another example in C: Is this simpler?
#include "checkit.h"
/* Demo of failure of branch coverage */
int foo(int a, int b)
{
int c;
if (a > 0)
{
c = a / 2;
}
else
{
c = 0;
}
if (b > 0)
{
b = b / c;
}
else
{
b = b * c;
}
return b;
}
int main()
{
/* The first two tests (together) create branch coverage */
checkit_int( foo(4, 6), 3); /* Execute TRUE branches */
checkit_int( foo(-2, -4), 0); /* Execute FALSE branches */
/* The third test reveals a defect */
checkit_int( foo(-2, 4), 4);
return 0;
}
a.
|
|
|
public class Item
{
private String item;
private int size;
public void setItem(String text)
{
item = text;
size =
item.length();
}
public void reduce()
{
if (size > 0)
{
item
= item.substring(1,size);
size
= size - 1;
}
}
public int getSize() { return size; }
public String getItem() { return item; }
}
|
Test Suite 8
|
Test Case
|
Pass?
|
a.
|
public void testOne()
{
Item item1 = new Item();
item1.setItem("grip");
assertEquals(4, item1.getSize());
assertEquals("grip", item1.getItem());
item1.reduce();
assertEquals(3, item1.getSize());
assertEquals("rip", item1.getItem());
}
|
|
b.
|
public void testTwo()
{
Item item1 = new Item();
item1.setItem("");
assertEquals(0, item1.getSize());
assertEquals("", item1.getItem());
item1.reduce();
assertEquals(0, item1.getSize());
assertEquals("", item1.getItem());
}
|
|
Number of statements executed by the suite (s): ______
Total number of statements (t): _______
% statement coverage (s/t): ________
Number of branches executed by the suite (b): ______
Total number of branches(n): _______
% branch coverage (b/n): ________
Number of paths executed by the suite (p): ______
Total number of paths (q): _______
% path coverage (p/q): ________
All our tests pass and we have 100% path coverage! Are we
done?
But what if?
public
void testThree()
{
Item item1 = new Item();
assertEquals(0, item1.getSize());
assertEquals("", item1.getItem());
}
The test fails, because in the second assert, getItem() returns null.
Next, read
the formal definitions for each of the previous examples of
coverage.