An efficient pool is critical for CAT administrations. Two approaches have been developed to design an optimal CAT pool: the linear programming method (LP; Veldkamp & van der Linden, 2000, 2010) and the bin-and-union method (BU; Reckase, 2003, 2010). This study manipulated different content balancing approaches and exposure conditions to investigate their impacts on the pool performances of the LP and BU methods under practical testing situations.
The optimal pools were constructed in terms of the specification of an operational fixed-length CAT program and the IRT model employed. This study considered the one-parameter logistic (1PL) model to simulate adaptive test item responses using optimal and operational pools. Several psychometric properties were compared between the pools designed under the LP and BU methods. This research attempted to answer the following question: Under the consideration of content balancing and exposure control, what were the benefits and limitations of the LP and BU methods with respect to the optimal pool design? The results were evaluated in terms of pool characteristics, content constraint management, item exposure control, pool utilization, test reliability, and measurement precision.
Similar pool characteristics were found between the LP and BU methods. With respect to the evaluation criteria, the LP and BU pools exhibited consistent performance. However, compared to the LP pools, the BU pools demonstrated slight superiority under the condition with strict content balancing and exposure control. Given two bin widths (.35 and .70), the pools with a bin-width of .35 exhibited better performance than those with a bin-width of .70 with respect to various evaluation criteria. Especially under the condition with the strict content balancing and exposure control, a bin-width of .35 might be a better option to generate an optimal pool than a bin-width of .70 in order to maintain a higher test on-target rate.