Dissertation
Evaluation of the goodness-of-fit index Mord in polytomous DCMs with hierarchical attribute structures
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Summer 2022
DOI: 10.25820/etd.006631
Abstract
The application of diagnostic classification models (DCMs) in the field of educational measurement is getting more attention in recent years. To make a valid inference from the model, it is important to ensure that the model fits the data. The purpose of the present study was to investigate the performance of the limited information goodness-of-fit statistic Mord in the polytomous response DCMs with the presence of hierarchical attribute structures. The first simulation study investigated the empirical Type I error rates of the Mord statistic under the null hypothesis when the model and data were perfectly fitted. The second simulation study explored the empirical rejection rate of Mord under different types of misspecifications: model misspecification, attribute hierarchy misspecification, and Q-matrix misspecification. The impact of test length, item quality, attribute structure, marginal/conditional probability of the mastery of attributes, and the number of response categories were investigated.
The results indicated that the Mord statistic demonstrated well-calibrated Type I error rates under the null conditions with different types of hierarchical attribute structures. When there were model-data misfits, the Mord statistic showed high empirical rejection rates in detecting the misspecified sDINA and sDINO but didn’t show enough power to detect the sC-RUM. The Mord also exhibited high empirical rejection rates when the generating attribute structure was non-strict hierarchical, but the fitted structure was strict hierarchical. Besides, it was sensitive when the sequence of parent and child attributes was reversed. However, the Mord statistic couldn’t detect the omission of hierarchical attribute connections. When there were Q-matrix misspecifications, the Mord statistic showed extremely high empirical rejection rates in the condition of Q-matrix under-specification but was not sensitive to the Q-matrix over-specification. The higher proportion of Q-matrix misspecification led to higher Mord rejection rates. The item quality exhibited a huge influence on the performance of the Mord statistic. The Mord statistic demonstrated higher empirical rejection rates with higher item quality, longer test length, fewer item response categories, and lower marginal/conditional probability of the mastery of attributes. The types of attribute structure also demonstrated slight influences on the rejection rates of the Mord statistics under some circumstances. The performance of the RMSEAord and SRMSR were explored in the present study to assess the degree of misfit and absolute model fit. The results indicated that the frequently used cut-off value of 0.05 was not appropriate in the framework of polytomous response DCMs. The magnitude of RMSEAord and SRMSR values varied in different situations.
Details
- Title: Subtitle
- Evaluation of the goodness-of-fit index Mord in polytomous DCMs with hierarchical attribute structures
- Creators
- Haimiao Yuan
- Contributors
- Jonathan Templin (Advisor)Boxiang Wang (Committee Member)Robert Ankenmann (Committee Member)Terry Ackerman (Committee Member)Won-Chan Lee (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Psychological and Quantitative Foundations (Educational Measurement and Statistics)
- Date degree season
- Summer 2022
- Publisher
- University of Iowa
- DOI
- 10.25820/etd.006631
- Number of pages
- xi, 132 pages
- Copyright
- Copyright 2022 Haimiao Yuan
- Language
- English
- Description illustrations
- illustrations, tables
- Description bibliographic
- Includes bibliographical references (pages 114-121).
- Public Abstract (ETD)
The present study aims to investigate the performance of the limited information goodness-of-fit statistic Mord when it was applied to the polytomous response diagnostic classification models (DCMs) with the presence of hierarchical attribute structures.
The results showed that the Mord statistic was appropriated to be applied to the polytomous response DCMs. When there were model-data misfits, the Mord showed good power in detecting the misspecified sDINA and sDINO model but was not sensitive in detecting the sC-RUM. It also exhibited high empirical rejection rates when the generating attribute structure was non-strict hierarchical, but the fitted structure was strict hierarchical. Besides, the Mord was sensitive to detecting the reversed hierarchical attribute connection but was not sensitive to the omission of connections. When there were Q-matrix misspecifications, the Mord demonstrated extremely high power in rejecting the Q-matrix under-specification but was not sensitive to the Q-matrix over-specification. The item quality, test length, attribute structure, number of response categories, and the mastery probability of the attributes exhibited influences on the empirical rejection rate of the Mord statistic. The values of the RMSEAord and SRMSR were also listed in the present study as a reference. The magnitude of RMSEAord and SRMSR values varied in different situations and the frequently used cut-off value of 0.05 was not appropriate in the framework of polytomous response DCMs.
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9984285247502771
Metrics
6 File views/ downloads
40 Record Views