Item Response Theory

12Test and Scale Development and Maintenance

The statistician who supposes that his main contribution to the planning of an experiment will involve statistical theory, finds repeatedly that he makes his most valuable contribution simply by persuading the investigator to explain why he wishes to do the experiment.

(Source: Gertrude Cox)

12.1 Introduction

This chapter covers a variety of topics related to the practical aspects of the use of item response theory (IRT) in the development and maintenance of tests and scales. We begin with an overview of item banking and calibration. Next, we discuss matrix item sampling using incomplete block designs. We follow with a discussion of test equating and conclude with a discussion of item parameter drift. Although there are large literatures related to many of these topics in the classical test theory (CTT) literature (particularly equating), our focus throughout is based on IRT (and various generalizations) to these problems because of their numerous advantages over CTT.

12.2 Item Banking

An item bank is a collection of items stored in an electronic format; i.e., a database. The item bank contains the text of the item, the answer categories, details regarding the development of the item, and various IRT parameters that describe the association between the item and the latent variable(s) of interest, and the item's difficulty or associated severity. There are several approaches to the development of an item bank. Most item banks ...