06 Black Box Testing

Tests are based on the specification of the function.

There are a bunch of advantages to that!

• By reading only the specification, the tester isn't influenced by the source code.
• Tests are more robust with respect to changes in implementation.
• Tests can be read and evaluated by reviewers who are not implementers

Main kinds of black box tests:

• Example inputs provided by specification
• Typical inputs
• Boundry cases
• Paths through a specification

Typical inputs

Common, simple values of a type. If you're writing a function that tests:

• int: small integers like 1 or 10
• char: alphabetic letters, digits
• string: whose lenght is a small number and whose characters are typical
• 'a list: a small integer number of elements, each of which is a typical value of type 'a
• records/tuples: each field/component with a typical value
• variants: typical constructors, if there is such a thing.

Boundry cases

QA Engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 999999999 beers. Orders a lizard. Orders -1 beers. Orders a sfdeljknesv.

— Bill Sempf (@sempf) September 23, 2014

Boundry testing involves testing boundary cases, also known as corner cases or edge cases. They are atypical or extremal values of a type and values.

• int: 0, 1, -1, min_int, max_int
• char: \000, \255, \032 (space), \127 (delete)
• string: empty string, string with a single character, unreasonably long string
• 'a list: empty list, list with a single element, list with enough elements to trigger a stack overflow on non-tail-recursive functions
• records/tuples: combinations of atypical values
• variants: all constructors

Paths through spec

Representitive inputs for all classes of outputs. For example, let's specify a function:

(** [is_prime n] is true iff [n] is prime *)
val is_prime: int -> bool

This function has two classes of output:

• true: representative input: n = 13
• false: representative input: n = 42

Other examples:

• compare functions have three classes of output (a > b, a < b, a = b)
• functions that return variants have several classes of output.

Representitive inputs for each way of satisfying the precondition.

(** [sqrt x n] is the square root of [x] computed to an
    accuracy of [n] significant figures.

    requires: x >= 0 and n >= 1 *)
val sqrt : float -> int -> float

There are four ways of satisfying the precondition (with respect to the >=)

• x = 0.0, n = 1
• x = 0.0, n = 2
• x = 1.0, n = 1
• x = 1.0, n = 2

Representitive inputs for each way of raising and not raising an exception

(** [pos x lst] is the 0-based position of the first element
    of [lst] that equals [x].

    raises: Not_found if [x] is not in [lst] *)
val pos : 'a -> 'a list -> int

• x = 1, lst = [1]
• x = 0, lst = [1]

Data abstractions

• Some functions of a data abstraction produce a value of it
  • empty
• Other functions consume a value of it
  • size
  • mem
• And others do both
  • add
  • remove

When testing data abstractions, test the interaction of producers and consumers

• size of empty
• size of add of empty