A comparison of various approaches to transitioning from linear tests to adaptive tests

For a long time, educational testing has taken the form of paper-and-pencil linear tests, meaning all the examinees, regardless of their abilities, take the same fixed test form. By contrast, adaptive testing sequentially selects the most appropriate items for individual examinees based on a dynamic ability estimation process during the test. In the past decades, many testing programs have completed or have started the transition from a linear testing format to an adaptive one to take the advantage of the more accurate shorter test produced by the adaptive testing. The first crucial question that is raised regarding the transition is what adaptive testing mode is to be adopted, given the purpose of the assessment.

The purpose of this study was to compare three major adaptive testing modes, including two traditional ones and one newly developed, with respect to measurement accuracy and item pool usage. The item pool consisted of items that were drawn from multiple linear forms of a large-scale achievement assessment administered in the past few years. The test specifications for the adaptive tests resembled those in the linear format to reflect the nature of transition. The comparisons were also conducted under reduced-size pool conditions.

Findings strongly recommended the newly developed method due to its flexibility in balancing measurement accuracy and item exposure, over the implementations of the traditional methods in the specific context of this study. However, the preference of the new method in this study should not be overgeneralized to other applications without an understanding of the purposes and contexts of them. This study provided useful information for the decision makers of the mode transition of this assessment. Also, it could be a reference for other testing programs who are considering such a transition in a similar context.