Step 7
Find Out How Consistent Scores Tend to Be
You will find information for this step in the technical material for the instrument. First, look for a section on reliability or, more specifically, sections on internal consistency, interrater reliability, and test-retest reliability.
Basically, reliability is consistency. There are three aspects of reliability covered here:
- homogeneity within scales
- agreement within rater groups
- stability over time
Without evidence of reliability, you do not know whether the items and scales of an instrument are good enough to hold up even under stable conditions with similar raters. In other words, you do not know whether items that are grouped together in a scale are measuring the same competency (homogeneity ...
Get Feedback to Managers: A Guide to Reviewing and Selecting Multirater Instruments for Leadership Development 4th Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
