Files and links (1)
Dissertation_draft4_Junhee Park_042726_submit2.58 MB
Embargoed Access, Embargo ends: 06/29/2027
Dissertation
Impact of integrating response time into item response theory models on educational assessment
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Spring 2026
DOI: 10.25820/etd.008339
Abstract
With the widespread adoption of computer-based testing in large-scale assessment, response time data have become available, providing insights into students’ test-taking behavior and test characteristics as sources of validity evidence. This study examines how integrating response time into item response theory (IRT) models affects the estimation of item and person parameters and, ultimately, how it influences test characteristics and ability inference compared to traditional IRT models.
Particularly, the distinction between timed and untimed testing conditions highlights how alignment between the intended construct and modeling approach affect parameter estimation and test characteristics. Two models were considered: a conventional two-parameter normal-ogive (2PNO) model using response accuracy only, and the hierarchical response time model proposed by van der Linden (2007), jointly modeling response accuracy and response time.
Three comparisons were conducted by using empirical dataset: (1) 2PNO versus hierarchical modeling under the timed condition, (2) 2PNO versus hierarchical modeling under the untimed condition, and (3) comparisons between timed and untimed tests. Results indicate that incorporating response time changed item and person parameter estimates, although strong linear relationships were observed between models. Across both testing conditions, the hierarchical model yielded higher test information, but reductions in the conditional standard error of measurement were minimal.
These findings highlight the importance of aligning the intended construct with the modeling framework and suggest that the incorporation of response time should be guided by considerations of construct representation and practical relevance. Implications for score reporting, item selection in computerized adaptive testing (CAT), and test administration are discussed.
Details
- Title: Subtitle
- Impact of integrating response time into item response theory models on educational assessment
- Creators
- Jun Hee Park
- Contributors
- Stephen B. Dunbar (Advisor)Catherine J. Welch (Committee Member)Deborah J. Harris (Committee Member)Won-Chan Lee (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Psychological and Quantitative Foundations
- Date degree season
- Spring 2026
- DOI
- 10.25820/etd.008339
- Publisher
- University of Iowa
- Number of pages
- x, 98 pages
- Copyright
- Copyright 2026 Jun Hee Park
- Language
- English
- Date submitted
- 04/27/2026
- Description illustrations
- tables, graphs
- Description bibliographic
- Includes bibliographical references (pages 94-98).
- Public Abstract (ETD)
In most large-scale educational assessments today, computers record not only whether test takers answer questions correctly but also how long they take to respond. This timing information has the potential to provide valuable insights into students’ test-taking behavior and how tests function. However, it is not yet clear whether incorporating response time provides changes in ability estimation across different testing conditions. Thus, further studies are needed to examine whether including response time changes how test takers’ ability is measured in assessments designed to measure ability alone versus those intended to measure both speed and ability.
Two models were compared using empirical data: a conventional method, the two-parameter item response theory (IRT) model based solely on accuracy, and a more advanced method, the hierarchical response time model, which analyzes both accuracy and response time together. The two models were compared across three scenarios: test with time limits, tests with unlimited time, and comparisons between the two. The findings show that including response time leads to some differences, but the results remain consistent in how student ability and item characteristics are estimated, particularly in the fifth-grade mathematics test.
Overall, the study emphasized the importance of aligning the intended construct with the modeling framework and suggested that the incorporation of response time should be guided by considerations of construct representation and practical relevance. Implications for score reporting, item selection in computerized adaptive testing (CAT), and test administration in K–12 educational contexts are discussed.
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985177270802771
Metrics
1 Record Views