I will begin with a distillation of our journey, a process for applying the scientific method to software engineering in an industrial environment. We called our version of the scientific method the Quality Improvement Paradigm (QIP) [Basili 1985], [Basili and Green 1994]. It consists of six basic steps:
Characterize the current project and its environment with respect to the appropriate models and metrics. (What does our world look like?)
Set quantifiable goals for successful project performance and improvement. (What do we want to know about our world and what do we want to accomplish?)
Choose the process model and supporting methods and tools for this project. (What processes might work for these goals in this environment?)
Execute the processes, construct the products, and collect, validate, and analyze the data to provide real-time feedback for corrective action. (What happens during the application of the selected processes?)
Analyze the data to evaluate the current practices, determine problems, record findings, and make recommendations for future project improvements. (How well did the proposed solutions work, what was missing, and how should we fix it?)
Package the experience in the form of updated and refined models and other forms of structured knowledge gained from this and prior projects, and save it in an experience base to be reused on future projects. (How do we integrate what we learned into the organization?)
The Quality Improvement Paradigm ...