Making Software

An Overview of Systematic Reviews

Believable research starts with primary studies: experiments with qualitative or quantitative results relevant to the question one is researching. An SR aggregates results from different independent experiments sometimes using statistical meta-analysis.

The classic example demonstrating the need for SRs comes from the medical domain. In 1990, Crowley et al. published an SR that included a meta-analysis of 12 primary studies on the effect of giving corticosteroids to pregnant women expected to give premature birth [Crowley et al. 1990]. Corticosteroids were believed to reduce lung problems affecting premature babies. Crowley at al.’s SR confirmed that the use of corticosteroids substantially decreased the risk of infant death. Corticosteroid administration was not the standard treatment for premature babies at the time, and the publication of the SR resulted in a change of medical practice.

However, this was not considered a major triumph for evidence-based medicine. Out of those 12 studies, 8 were published before 1982. If those 8 studies had been aggregated in 1982, eight years of incorrect treatment and associated infant deaths would have been avoided. This led to a reassessment of the importance of aggregating evidence in a timely manner and to the establishment of the Cochrane Collaboration, a nonprofit organization that undertakes SRs of medical and health care issues and maintains a database of SR reports.

You can find an update of Crowley et ...