Data Analysis and Applications 1

14Efficiency Evaluation of Multiple-Choice Questions and Exams

Multiple-choice questions are common in Israeli institutions of higher education. They can be checked and graded automatically using artificial intelligence methods so that the answer sheets are aligned and segmented automatically into the relevant regions, and then the answers marked by the students are read. In the next step, the grades can be easily calculated by comparing the marked data with the correct answers. To evaluate the efficiency of the exam in addition to the basic statistical analysis of the grades, we propose efficiency measures for each question as well as for the whole exam. These efficiency measures attempt to answer the following questions: how many of the “strong” students have answered a particular question correctly and how many of the “weak” students have failed in a particular question. A question is considered efficient if most “strong” students succeed in it, whereas most “weak” ones fail. In a similar fashion, an exam questionnaire is considered efficient if the majority of its questions are efficient. Our measures can be used both for multiple-choice and numeric answers. We have performed the proposed statistical analysis on the grades of a number of real-life examinations, and our conclusion is that the proposed analysis and efficiency measures are beneficial for the purpose of estimating the quality of the exam and discovering the inefficient questions, that is, the ones that fail ...