The Cleanroom Development Process

Developed by Harlan Mills (then with IBM), Cleanroom seeks an analogy in software development with the processes of hardware production. The process depends on strict separation of code creation and testing tasks, and development of accurate profiles of expected client usage of a product.

No unit testing... verification instead

In Cleanroom, no unit testing is done. Instead, units are formally verified using Mills' own functional techniques. Mills verification is a cookbook approach to denotational semantics, treating programs as mappings from input domains to output domains. Unit programmers work with functional specs, and when code is developed they verify it manually with functional Mills proofs (we did some of these in COMP 204). Unit programmers are given parsers for the source language, but they do not generate and run any target code.

Integration testing on random inputs

After units are verified they are sent to a separate team for compilation and assembly into a system, and then for integration testing. Unit developers do not do any integration testing; they are completely separated from running their code. When an integrated system is assembled, testing is done by drawing data at random from a probability distribution of most likely inputs. These statistical profiles are developed by the system integration team in conjunction with the client. The client is interviewed and asked how the product is going to be used in practice.

Quality is in the eyes of the client

Since the data profiles reflect input probilities for actual (expected) client use, the errors found during random testing are then the errors most likely to occur had the client actually run the program in the expected usage environment. If you find the errors most likely to occur when a client uses the program, then the client is less likely to find errors when testing is complete, and the client is more likely to conclude that the program is well-built. The process is obviously sensitive to errors in the usage profiles. The success of random testing is dependent on the accuracy of the statistical data distributions developed with the client.