Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice

Walter Vispoel; Hyeryung Lee; Tingting Chen

doi:10.3390/math12081164

Back

Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice

Journal article

Open access

Peer reviewed

Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice

Walter Vispoel, Hyeryung Lee and Tingting Chen

Mathematics (Basel), Vol.12(8), 1164

04/01/2024

DOI: 10.3390/math12081164

Files and links (1)

url

https://doi.org/10.3390/math12081164View

Published (Version of record) Open Access

Abstract

We illustrate how structural equation models (SEMs) can be used to assess the reliability and generalizability of composite and subscale scores, proportions of multiple sources of measurement error, and subscale added value within multivariate designs using data from a popular inventory measuring hierarchically structured personality traits. We compare these techniques between standard SEMs representing congeneric relations between indicators and underlying factors versus SEM-based generalizability theory (GT) designs with simplified essential tau-equivalent constraints. Results strongly emphasized the importance of accounting for multiple sources of measurement error in both contexts and revealed that, in most but not all instances, congeneric designs yielded higher score accuracy, lower proportions of measurement error, greater average subscale score viability, stronger model fits, and differing magnitudes of disattenuated subscale intercorrelations. Extending the congeneric analyses to the item level further highlighted consistent weaknesses in the psychometric properties of negatively versus positively keyed items. Collectively, these findings demonstrate the practical value and advantages of applying GT-based principles to congeneric SEMs that are much more commonly encountered in the research literature and more directly linked to the specific measures being analyzed. We also provide prophecy formulas to estimate reliability and generalizability coefficients, proportions of individual sources of measurement error, and subscale added-value indices for changes made to measurement procedures and offer guidelines and examples for running all illustrated analyses using the lavaan (Version 0.6-17) and semTools (Version 0.5-6) packages in R. The methods described for the analyzed designs are applicable to any objectively or subjectively scored assessments for which both composite and subcomponent scores are reported.

Accuracy

Error analysis

Multivariate analysis

Reliability analysis

Variance analysis

Details

Title: Subtitle: Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice
Creators: Walter Vispoel - University of Iowa
Hyeryung Lee - University of Iowa
Tingting Chen - University of Iowa
Resource Type: Journal article
Publication Details: Mathematics (Basel), Vol.12(8), 1164
Publisher: MDPI AG
DOI: 10.3390/math12081164
eISSN: 2227-7390
Language: English
Date published: 04/01/2024
Academic Unit: Psychological and Quantitative Foundations
Record Identifier: 9984621138002771

Metrics

5 Record Views