Dissertation
mstATA: an R package for IRT-based multistage test assembly
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Spring 2026
DOI: 10.25820/etd.008382
Abstract
Multistage testing (MST) has gained prominence as an adaptive testing design that balances measurement efficiency with operational simplicity. However, assembling MST panels that satisfy complex specifications and psychometric targets remains challenging. Although automated test assembly (ATA) methods have advanced, there is still a lack of flexible, transparent tools tailored to the structural complexities of MST. Existing mixed-integer linear programming (MILP)-based tools primarily focus on linear test assembly and provide limited support for MST designs.
This dissertation introduces mstATA, a transparent framework and user-friendly R package that extends hybrid assembly strategies to address these limitations. The framework adopts a unified definition of binary decision variables applicable to top-down, bottom-up, and extended hybrid approaches, supporting panels composed of standalone items, stimulus-based items, or both. A core principle of the framework is the distinction between item pool attributes and test specification requirements. Item pool attributes are organized at the item, stimulus, and item-set levels, including categorical, quantitative, and logical characteristics. Test specifications are defined across multiple hierarchical levels, including item, stimulus, item set, module, pathway, panel, and solution levels.
The framework enforces logical consistency across these levels, including item–stimulus conditional selection, whereby items linked to an unselected stimulus cannot be selected, and cross-panel consistency, whereby an item selected in any panel must have a solution-level indicator equal to one, and a zero indicator precludes selection across all panels. Beyond standard optimization approaches (maximin, capped maximin, and minimax), mstATA incorporates weighted-sum and goal-programming formulations, enabling flexible multi-objective optimization.
Notably, infeasibility diagnosis and model reformulation are treated as integral components of the framework, enabling identification of conflicting requirements and supporting principled specification refinement. In addition, the package provides efficient reporting and evaluation of panel characteristics and measurement precision using analytic methods that avoid computationally intensive Monte Carlo simulation. By bridging methodological advances with practical implementation, mstATA offers a robust solution for modern assessment programs.
Details
- Title: Subtitle
- mstATA: an R package for IRT-based multistage test assembly
- Creators
- Hong Chen
- Contributors
- Catherine Welch (Advisor)Stephen Dunbar (Advisor)Aixin Tan (Committee Member)Anthony Fina (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Psychological and Quantitative Foundations (Educational Measurement and Statistics)
- Date degree season
- Spring 2026
- DOI
- 10.25820/etd.008382
- Publisher
- University of Iowa
- Number of pages
- xviii, 232 pages
- Copyright
- Copyright 2026 Hong Chen
- Language
- English
- Date submitted
- 04/23/2026
- Description illustrations
- graphs, tables
- Description bibliographic
- Includes bibliographical references (pages 219-232).
- Public Abstract (ETD)
Multistage testing (MST) has become an increasingly widely used approach for delivering large-scale assessments. MST is a form of adaptive testing in which the difficulty of subsequent test modules is adjusted based on examinee’s performance on previous stages. Although this adaptive structure improves measurement efficiency and test experience, constructing MST exams remains a complex task, as test developers must simultaneously satisfy strict content constraints and psychometric targets.
This dissertation introduces mstATA, a software tool implemented as an R package to support automated MST assembly. In contrast to existing tools that are often designed for linear tests or lack flexibility, mstATA is specifically developed to address the structural and logical complexities of MST design. The framework supports diverse item types, including stimulus-based items, and facilitates the construction of multiple parallel test forms while maintaining consistency and fairness across panels.
A key feature of mstATA is its diagnostic capability for infeasibility. When a set of specifications cannot be satisfied simultaneously, the tool identifies the sources of conflict and supports systematic revision of constraints. In addition, mstATA provides efficient analytic reporting of measurement characteristics without relying on computationally intensive simulation procedures.
Overall, mstATA bridges advanced optimization methodologies with practical test development needs, offering a transparent and flexible tool for the design of modern multistage assessments.
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985177272402771
Metrics
1 Record Views