Making Software

The Strengths and Weaknesses of Systematic Reviews

I have defined, over time, two standards for performing SRs suitable for software engineering researchers. The first standard was based on medical SRs [Kitchenham 2004], whereas the second revised the first standard to take into account new ideas arising from sociological research [Kitchenham and Charters 2007].^[9]

This section provides a brief summary of the guidelines documented in these reports. However, I caution readers not to assume they can successfully undertake an SR based on this brief overview. You should read the full technical reports, consult other references (for example, [Fink 2005], [Higgins and Green 2009], [Khan et al. 2003], [Petticrew and Roberts 2006]) and read examples of SRs, including the software engineering examples discussed later in this chapter. If you look at some good examples of SRs, you will appreciate that they are not an easy form of research. They take a long time and require meticulous planning, execution, and reporting [Brereton et al. 2007].

The Systematic Review Process

An SR has three main phases: planning, conducting the review, and reporting the results.

Planning the review

The stages of this phase are:

Identifying the need for a review

Identification of an information need helps to define the context for the review and may place constraints on the research questions or the criteria used to include and exclude primary studies. For example, if you need to determine the best way to introduce a ...