Categorical Data Analysis Using The SAS® System, 2nd Edition

5.5. Observer Agreement

5.5.1. Computing the Kappa Statistic

For many years, researchers in medicine, epidemiology, psychiatry, and psychological measurement and testing have been aware of the importance of observer error as a major source of measurement error. In many cases, different observers, or even the same observer at a different time, may examine an x-ray or perform a physical examination and reach different conclusions. It is important to evaluate observer agreement, both to understand the possible contributions to measurement error and as part of the evaluation of testing new instruments and procedures.

Often, the data collected as part of an observer agreement study form a contingency table, where the column levels represent the ...