07 Glass Box Testing

Advantages:

• can determine whether a new test case really yields any additional information about the correctness of an implementation
• can addfress likely errors that are not apparent from specification

Glass box testing supplements, it does not replace black box testing.

In glass box testing, our goal is to cover the entire program in test cases. Ensure that the entire program is excercised by tests

• Branches (if, match, Boolean ops, exceptions, loops, etc.) make this challenging
• The exact definition of coverage is flexible.
  • | Kind | Meaning | |-|-| | Statement | Evaluate every statement/expression | | Condition | Evaluate every boolean/pattern match to each possible value | | Path | Causes every possible execution path through program to occur |

Let's make this concrete!

let max3 x y z =
	if x > y then
		if y > z then x else z
	else
		if y > z then y else z

val max3 : 'a -> 'a -> 'a -> 'a = <fun>

There are only 4 paths this function can take with it's if expressions. Testing along all black-box specifications might lead to all kinds of input, but for glass box testing, we can test with

• max3 3 2 1
• max3 3 2 4
• max3 1 2 1
• max3 1 2 3

We then hit every condition in the function! But we haven't really tested against the implementation, so black-box testing is still nessicary.

Include test cases for:

• each branch of each (nested) if expression
• each branch of each (nested) pattern match

Watch out for:

• base cases of recursive functions
• recursive calls in recursive functions
• every place where an exception might be raised

Let's write a function that determins if a year is a leap year.

(** [leap_year y] is whether [y] is a
	leap year in the Gregorian calendar
	
	Requires: [y >= 1] *)
let leap_year y =
	y mod 4 = 0
	&&
	(y mod 100 <> 0 || y mod 400 = 0)

val leap_year : int -> bool = <fun>

Suppose we wrote the test suite with these inputs

• 2000
• 2010
• 2020

What kind of coverage does the test suite achieve?

	y mod 4 = 0	y mod 100 <> 0	y mod 400 = 0
2000	true	false	true.
2010	false	Not evaluated.	Not evaluated.
2020	true	true	Not evaluated.

-Statement coverage is acheived by just 200- -Condition coverage is not achieved. No input causes y mod 400 = 0 to evaluate to false. We can add the test case for 2100 to do that!

	y mod 4 = 0	y mod 100 <> 0	y mod 400 = 0
2100	true	false	false.