Journal article
A Closer Look at Using Judgments of Item Difficulty to Change Answers on Computerized Adaptive Tests
Journal of educational measurement, Vol.42(4), pp.331-350
12/2005
DOI: 10.1111/j.1745-3984.2005.00019.x
Abstract
Recent studies have shown that restricting review and answer change opportunities on computerized adaptive tests (CATs) to items within successive blocks reduces time spent in review, satisfies most examinees' desires for review, and controls against distortion in proficiency estimates resulting from intentional incorrect answering of items prior to review. However, restricting review opportunities on CATs may not prevent examinees from artificially raising proficiency estimates by using judgments of item difficulty to signal when to change previous answers. We evaluated six strategies for using item difficulty judgments to change answers on CATs and compared the results to those from examinees reviewing and changing answers in the usual manner. The strategy conditions varied in terms of when examinees were prompted to consider changing answers and in the information provided about the consistency of the item selection algorithm. We found that examinees fared best on average when they reviewed and changed answers in the usual manner. The best gaming strategy was one in which the examinees knew something about the consistency of the item selection algorithm and were prompted to change responses only when they were unsure about answer correctness and sure about their item difficulty judgments. However, even this strategy did not produce a mean gain in proficiency estimates.
Details
- Title: Subtitle
- A Closer Look at Using Judgments of Item Difficulty to Change Answers on Computerized Adaptive Tests
- Creators
- Walter P. Vispoel - University of IowaSara J. Clough - University of IowaTimothy Bleiler - University at Buffalo, State University of New York
- Resource Type
- Journal article
- Publication Details
- Journal of educational measurement, Vol.42(4), pp.331-350
- Publisher
- Blackwell Publishing
- DOI
- 10.1111/j.1745-3984.2005.00019.x
- ISSN
- 0022-0655
- eISSN
- 1745-3984
- Number of pages
- 20
- Language
- English
- Date published
- 12/2005
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9984371295402771
Metrics
12 Record Views