Pretest item calibration in multistage adaptive testing

In multi-stage adaptive testing (MST), examinees take items sets of different difficulty based on their proficiency levels. This design allows for more precise and efficient measurement over a wider range of proficiency. However, a relatively larger item bank is required to measure a broad range of proficiency. At the same time, item security and quality must be maintained by the continuous addition of new items. These new items must be placed on the scale of existing items within the item bank via various calibration and linking methods. The purpose of this dissertation is to evaluate for MST designs the quality of item parameter recovery (i.e., accurate estimation of item parameters) resulting from selected calibration and linking methods used for placing pretest item parameter estimates on the scale of the item pool.

Pretest items can be administered as part of the operational administration, with varied placement of pretest items within the MST design. Under the embedded-section (ES) model, all pretest items were administered in a separate module. With the embedded-items (EI) model, pretest items were distributed across modules.

Using all of the operational items for linking, even though there were missing responses, provided the most accurate parameter estimates under both separate (SC) and fixed parameter calibration (FC). The SC approach recovered the item parameters better than FC, even though the discrepancy between them was small. Also, SC appeared to be more stable than FC across different study conditions. Between pretesting models, ES was more stable than EI across varied study conditions.