White Papers / Thought Leadership

Achieving High-Quality Software Systems: A Comprehensive Approach to Testing and Validation
Copyright © 2004 by the President and Fellows of Harvard College. Posted with permission of Harvard University
Charles Popper
March 2004

Abstract: In the twenty-first century, when software has become a key force in daily life and its malfunctioning can threaten the public health, safety, and economic well-being, the challenge is to ensure that the quality of software systems is the highest possible. This report presents an approach to analyzing poor quality software systems, by examining their effects, the nature of their defects, and the causes of these defects. A broad theory of quality management is applied to evaluating the quality of software, and broad concepts of total quality management and six sigma are related and applied along with such concepts such as the prevention and detection of defects and estimations of reliability.
Four principles of high-quality software are developed. The first principle is that metrics for the evaluation of quality and other, associated targets need to be defined for each stage of the development life cycle for software. Second, a method for the management of quality is essential to keep the process of improvement going forward. Third, experience, expertise, and training in the planning and use of testing procedures focused on quality also are essential. And, fourth, whether or not a software system meets its requirements should be determined by independent agents-neither the developers, who were paid to produce the systems, nor the technology vendors, who sell off-the-shelf products, have the objectivity required for impartial review.
The report concludes with an analysis of the benefits of a good, comprehensive program for testing the quality of a software program, in particular, the benefits of using independent experts to manage the quality of the program.

click here to view / download the full published version of Achieving High-Quality Software Systems