04 Approaches to Testing

In testing, our goal is to expose the existence of faults so that they can be fixed and not lead to failures

There are many different methods of testing

• Unit testing: isolated components
• Integration testing: combined components
• System testing: functionality, performance, acceptance, installation

In CS 3110, we're mostly going to talk about unit testing.

Another component of testing is regression testing

• Regression: a previously fixed fault is reintroduced into the code
• Regression testing: running tests against new versions of software to ensure that no regressions have occured
• If you ever find and fix a fault
  • Put a test case in your suite for it
  • Run suite frequently to detect regresions.

Fun fact! Does the probabiliy of undetected faluts increase or decrease with detected faults. It increases (Myers 1979, 2004)!

When do you stop testing?

• Bad answer: when time is up
• Bad answer: when all tests pass
• Bad answer: when the rubric says you're good
• Good answer: when testing methodology is complete
• Future answer: when statistical estimation says that the probability of undetected faults is low enough