Detecting attribute and measurement invariance in the log-linear cognitive diagnostic model in the presence of attribute hierarchies

Diagnostic assessments are a powerful means for understanding what students know and can do. Whereas traditional educational assessments provide educational professionals with aggregate-level information (e.g., performance of the classroom as a whole), diagnostic assessments are designed to provide student-level information and are often used to provide teachers with a rich, comprehensive profile that outlines their student’s strengths and weaknesses and can be used to provide recommendations for improvement on areas of deficiency. Despite the increasing popularity of diagnostic assessments, statistical methods that can gauge the psychometric properties of such assessments across multiple populations are needed, especially as issues surrounding fairness and equity in educational assessments begin to rise.

The primary purpose of this thesis is to develop a psychometric framework for statistically detecting invariance – or fairness – in diagnostic assessments. The framework developed in this work provides a natural way for testing if the skills measured by an assessment are related to and interact with each other in the same way across multiple groups (attribute invariance) and if the tasks (items) on the assessment function in the same way across group multiple groups (measurement invariance). An application of the proposed framework to an empirical dataset demonstrates that the framework is capable of detecting invariance at the attribute and item level.