4. Test

"Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence." — Edsger W. Dijkstra

"The real value of tests is not that they detect bugs in the code, but that they detect inadequacies in the methods, concentration, and skills of those who design and produce the code." - C. A. R. Hoare

Heavily depends on the Problem Reinterpreting and Design Phases

1. Skeptic Mindset

1. Be your own Devil's Advocate
2. Be Hacker - The Goal is to prove that the code is broken, not that it works
3. If you don't test a piece of code, you cannot trust it and you cannot depend upon it
4. Re-run old tests after every new change

2. Programmer Responsibility - Validate the Program's Correctness

3. Test Case Layers

1. Feature / Scenario - Using the language of the problem phase/domain
2. Instance - Concrete Input-Output pairs

4. Testing Principle - Simplicity

The instance(s) should be the simplest possible that test the scenario.

5. Test Case Types

1. By Perspective

   1. External - Black-Box (~ By Data)
   2. Internal - White-Box / Glass-Box (~ By Flow)

2. By Data (Type and Value) ¹
   Note: Not all the test case types are applicable for every program.

   1. Sanity Check
   2. Trivial Cases - The work to be done is trivial or nonexistent
      1. Sub-case: Already Solved Cases - Nothing needs to be modified
   3. Representational Cases
      1. Simplest non-Trivial Cases
      2. General Cases
   4. Edge Cases - extreme normal operating parameters
   5. Corner Cases - outside normal operating parameters
      1. Sub-case: Input Validation

3. By Flow - Coverage / Path-Complete - All possible code flows (branches)

   1. Conditionals (including implied-else)
   2. Loops (zero, one, multiple and all iterations)

4. By Scope

   1. Unit Tests
   2. Regression Tests
   3. Integration Tests
   4. System Tests

6. Bug Types

1. Syntax Error (including IndentationError)
2. Runtime Error (Exceptions)
3. Semantic/Logic Error

7. Writing Tests

1. When?
   1. Before coding (the BEST!)
   2. After coding (not so good)
   3. While coding - Interactively - doctest
2. How?
   1. assert
   2. doctest
   3. Unit Testing Frameworks

Checkout Question:

Do I know all the Inputs of which the code doesn't return the desirable Output?

¹ Empty Input - Depends on the program, the empty input may be part of each one of the test cases types.