Journal article
A Comparative Study of On-line Pretest Item-Calibration/Scaling Methods in Computerized Adaptive Testing
Journal of educational measurement, Vol.38(3), pp.191-212
09/2001
DOI: 10.1111/j.1745-3984.2001.tb01123.x
Abstract
The purpose of this study was to compare and evaluate five on-line pretest item-calibration/scaling methods in computerized adaptive testing (CAT): marginal maximum likelihood estimate with one EM cycle (OEM), marginal maximum likelihood estimate with multiple EM cycles (MEM), Stocking's Method A, Stocking's Method B, and BILOG/Prior. The five methods were evaluated in terms of item-parameter recovery, using three different sample sizes (300, 1000 and 3000). The MEM method appeared to be the best choice among these, because it produced the smallest parameter-estimation errors for all sample size conditions. MEM and OEM are mathematically similar, although the OEM method produced larger errors. MEM also was preferable to OEM, unless the amount of time involved in iterative computation is a concern. Stocking's Method B also worked very well, but it required anchor items that either would increase test lengths or require larger sample sizes depending on test administration design. Until more appropriate ways of handling sparse data are devised, the BILOG/Prior method may not be a reasonable choice for small sample sizes. Stocking's Method A had the largest weighted total error, as well as a theoretical weakness (i.e., treating estimated ability as true ability); thus, there appeared to be little reason to use it.
Details
- Title: Subtitle
- A Comparative Study of On-line Pretest Item-Calibration/Scaling Methods in Computerized Adaptive Testing
- Creators
- Jae-Chun Ban - ACTBradley A. Hanson - CTB/McGraw-HillTianyou Wang - ACTQing Yi - ACTDeborah J. Harris - ACT
- Resource Type
- Journal article
- Publication Details
- Journal of educational measurement, Vol.38(3), pp.191-212
- Publisher
- Blackwell Publishing Ltd
- DOI
- 10.1111/j.1745-3984.2001.tb01123.x
- ISSN
- 0022-0655
- eISSN
- 1745-3984
- Number of pages
- 22
- Language
- English
- Date published
- 09/2001
- Academic Unit
- Center for Advanced Studies in Measurement and Assessment; Psychological and Quantitative Foundations
- Record Identifier
- 9984627201002771
Metrics
1 Record Views