Journal article
The impact of design decisions on measurement accuracy demonstrated using the Hierarchical Rater Model
Psychological test and assessment modeling, Vol.59(4), pp.471-492
10/01/2017
Abstract
When humans assign ratings in testing contexts, concern arises about whether rater effects impact the accuracy of the resulting measures. Those who lead scoring efforts implement several activities and utilize various designs to minimize the impact of these rater errors. This article uses the Hierarchical Rater Model (HRM) to demonstrate how the magnitude of rater errors and numbers of ratings associated with various measurement facets (e.g., raters & items) impact the accuracy of measures. Additionally, we demonstrate how the level at which decisions are made about the measures (e.g., test taker item scores, test taker total scores, test taker classifications) impact measurement accuracy.
Details
- Title: Subtitle
- The impact of design decisions on measurement accuracy demonstrated using the Hierarchical Rater Model
- Creators
- Jodi CasabiancaEdward Wolfe
- Resource Type
- Journal article
- Publication Details
- Psychological test and assessment modeling, Vol.59(4), pp.471-492
- ISSN
- 2190-0493
- eISSN
- 2190-0507
- Publisher
- PABST Science Publishers
- Language
- English
- Date published
- 10/01/2017
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985134647002771
Metrics
1 Record Views