02 Testing and Validation

• Validation: does a program behave as intended
• Testing: a process for validation?
• Debugging: determining the cause of unintended behavior
• Defensive programming: implementation techniques to make validation and debugging easier

There are many approaches to validation

• Social
  • Code review
  • Extreme/Pair Programming
• Methodological
  • Test-driven development
  • Version control
  • Bug tracking
• Technological
  • Static analysis (linters, FindBugs, ...)
  • Fuzzers
• Mathematical
  • Type systems
  • Formal verification (CS 4160)

This is a spectrum! The methods at the top are less formal, and may miss problems, and the bottom are more formal! Even the most formal methodologies have holes though. Even if you mathematically prove that the program is correct, you must ask... "did I prove the right thing?"

Testing vs. Verification

Testing

• Cost effective
• Guarantee that program is correct on tested inputs and in tested enivorments

Verification

• Expensive (requires training in advanced computer science and math techniques)
• Guarantee that program is correct on all inputs in all enviorments.

Verification is traditionally used for high stakes enviorments, such as radiation machines, aeronatics, and spaceflight. We'll study a little verification later in the course.

Program testing can at best show the presence of errors, but never their abscence.

Edsger W. Dijkstra, 1972 Turing Award winner