Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs

Walter P. Vispoel; Hyeryung Lee; Tingting Chen

doi:10.3390/math13061001

Back

Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs

Journal article

Open access

Peer reviewed

Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs

Walter P. Vispoel, Hyeryung Lee and Tingting Chen

Mathematics (Basel), Vol.13(6), 1001

03/19/2025

DOI: 10.3390/math13061001

Files and links (1)

url

https://doi.org/10.3390/math13061001View

Published (Version of record) Open Access

Abstract

Generalizability theory (GT) provides an all-encompassing framework for estimating accuracy of scores and effects of multiple sources of measurement error when using measures intended for either norm- or criterion-referencing purposes. Structural equation models (SEMs) can replicate results from GT-based ANOVA procedures while extending those analyses to account for scale coarseness, generate Monte Carlo-based confidence intervals for key parameters, partition universe score variance into general and group factor effects, and assess subscale score viability. We apply these techniques in R to univariate, multivariate, and bifactor designs using a novel indicator-mean approach to estimate absolute error. When representing responses to items from the shortened form of the Music Self-Perception Inventory (MUSPI-S) using 2-, 4-, and 8-point response metrics over two occasions, SEMs reproduced results from the ANOVA-based mGENOVA package for univariate and multivariate designs with score accuracy and subscale viability indices within bifactor designs comparable to those from corresponding multivariate designs. Adjusting for scale coarseness improved the accuracy of scores across all response metrics, with dichotomous observed scores least approximating truly continuous scales. Although general-factor effects were dominant, subscale viability was supported in all cases, with transient measurement error leading to the greatest reductions in score accuracy. Key implications are discussed.

structural equation models

generalizability theory

multivariate analysis

bifactor models

R programming

reliability

validity

subscale viability

Music Self-Perception Inventory

<italic>mGENOVA</italic>

62-P25

Details

Title: Subtitle: Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs
Creators: Walter P. Vispoel - University of Iowa
Hyeryung Lee - University of Iowa
Tingting Chen - University of Iowa
Resource Type: Journal article
Publication Details: Mathematics (Basel), Vol.13(6), 1001
Publisher: MDPI
DOI: 10.3390/math13061001
ISSN: 2227-7390
eISSN: 2227-7390
Grant note: Iowa Testing Programs
This project received no external funding but did receive internal research assistant support from the Iowa Testing Programs.
Language: English
Date published: 03/19/2025
Academic Unit: Psychological and Quantitative Foundations
Record Identifier: 9984802103302771

Metrics

1 Record Views