Class Meeting: Testing

Admin

• Implementation Manual due soon

Notes

Systematic Testing

• planned: to permit design for testability
• documented: so test cases can be easily understood and the adequacy of the test cases evaluated
• maintained: so that test cases can be executed after every change to the software

Testing tasks

1. build the test harness
2. generate the test inputs
3. determine the expected outputs for each of the test inputs
4. execute the test cases, monitoring the behavior of the program
5. compare the actual outputs to the expected outputs
6. evaluate the test results and decide whether the program needs more repair/development

The Test Plan

There are 4 sections to TP:

• assumptions
• test environment (support for testing, scaffolding, etc.)
• test case selection strategy
• test implementation strategy

The Test Implementation

Here is a sample test driver:

#define QUIT 0
#define S_INIT 1
#define S_PUSH 2
#define S_POP 3
#define G_TOP 4
#define G_DEPTH 5
#define G_DUMP 6

main()
{
  int reply, i;

  while ((reply=nextcall()) != QUIT) {
    switch(reply) {
      case S_INIT:
        ps_s_init();
        break;
      case S_PUSH:
        i = readint("Enter element:");
        ps_s_push(i);
        break;
      case S_POP:
        ps_s_pop();
        break;
      case G_TOP:
        i = ps_g_top();
        printf("returns %d\n",i);
        break;
      case G_DEPTH:
        i = ps_g_depth();
        printf("returns %d\n",i);
        break;
      case G_DUMP:
        ps_g_dump();
        break;
    }
  }
  return(0);
}

Issues

• Unit testing vs. Integration testing
• Test case generation/selection
• Automation
• Interactive/GUI subsystems
• code inspections (can be more general in error detection)

Testing Approaches

• black box vs. white box
  • cc -a and tcov (try on baldhead)
• show path analysis forms of white box
• show functional testing with aglebraic specs
• Error oriened testing (mutation, seeding)
• Clean room: statistical (random) testing

System Integration

• Top-down integration strategy
  Implement top-level modules first, provide stubs to imitate lower-level services to make a system for testing

• Bottom-up integration strategy
  develop lower-level modules first, provide drivers to invoke them to make a system for testing.