Report
Conditional Standard Errors, Reliability and Decision Consistency of Performance Levels Using Polytomous IRT
p.22 pages
Distributed by ERIC Clearinghouse
04/1996
Abstract
M. J. Kolen, B. A. Hanson, and R. L. Brennan (1992) presented a procedure for assessing the conditional standard error of measurement (CSEM) of scale scores using a strong true-score model. They also investigated the ways of using nonlinear transformation from number-correct raw score to scale score to equalize the conditional standard error along the reported score scale. Kolen, L. Zeng, and Hanson (in press) presented a similar procedure for assessing CSEM using item response theory (IRT) techniques. This paper extends that procedure to tests with polytomous items using a polytomous IRT model approach. A polytomous IRT-based procedure to assessing decision consistency of performance level classification based on alternate test forms is also described. The general approach for assessing CSEM and reliability is to obtain the probability distribution of the level score conditioned on a given theta and then compute the conditional mean and conditional standard deviation (or variance) of the scale scores or the level scores. The CSEM of the level score is the conditional standard deviation. Data from the American College Testing Program's Work Keys assessment for 7,097, 2,035, and 1,793 samples illustrate the use of the procedures. Model fit, classification consistency, and reliability were evaluated and found acceptable. Results suggest that the new level scores have higher reliability and classification consistency than the old level scores, indicating the usefulness of these polytomous IRT-based procedures. (Contains 4 tables, 5 figures, and 16 references.) (SLD)
Details
- Title: Subtitle
- Conditional Standard Errors, Reliability and Decision Consistency of Performance Levels Using Polytomous IRT
- Creators
- Tianyou Wang
- Resource Type
- Report
- Publication Details
- p.22 pages
- Publisher
- Distributed by ERIC Clearinghouse
- Comment
- ERIC Number: ED401323
- Language
- English
- Date published
- 04/1996
- Academic Unit
- Center for Advanced Studies in Measurement and Assessment
- Record Identifier
- 9984632146702771
Metrics
1 Record Views