Generalizability Theory's Role in Validity Research: Innovative Applications in Health Science Education

Clarence Kreiter; Nikki Bibler Zaidi

doi:10.1016/j.hpe.2020.02.002

Back

Generalizability Theory's Role in Validity Research: Innovative Applications in Health Science Education

Journal article

Open access

Peer reviewed

Generalizability Theory's Role in Validity Research: Innovative Applications in Health Science Education

Clarence Kreiter and Nikki Bibler Zaidi

Health professions education, Vol.6(2), pp.282-290

06/2020

DOI: 10.1016/j.hpe.2020.02.002

Files and links (1)

url

https://doi.org/10.1016/j.hpe.2020.02.002View

Published (Version of record) Open Access

Abstract

While generalizability (G) theory is widely recognized as a method for estimating the reliability (precision) of measures, its unique approach to partitioning and quantifying variance also yields validity (accuracy) evidence. Yet, G theory’s ability to provide validity evidence is much less understood and established in the literature. The purpose of this paper is to demonstrate G theory’s potential for addressing a wide array of health sciences education validity questions. Using Kane’s validity framework, this paper explores the use of G theory in the health sciences literature by presenting a number of validity applications. The G studies investigate validity-related measurement questions and demonstrate how G theory contribute to one or more of the four types of Kane’s validity inferences (scoring, generalization, extrapolation, and implication). Each G study is linked to one of Kane’s four types of validity inferences. The studies presented in this paper demonstrate how a G theory analysis of score variance, usually within existing (in vivo) assessment data, simultaneously provides researchers with evidence regarding both reliability and validity and offers a more accurate portrayal of the relationship between the two. Because each application of G theory is unique, the examples provided do not represent the entire range of potential applications, but rather demonstrate the methodological flexibility of G theory in addressing complex validity questions. Further advances will require researchers to develop and share additional innovations in G study design and work together to develop a consensus regarding its role in validity research.

Assessment

Generalizability theory

Methodology

Reliability

Validity

Details

Title: Subtitle: Generalizability Theory's Role in Validity Research: Innovative Applications in Health Science Education
Creators: Clarence Kreiter - Roy J. and Lucille A. Carver College of Medicine
Nikki Bibler Zaidi - University of Michigan
Resource Type: Journal article
Publication Details: Health professions education, Vol.6(2), pp.282-290
DOI: 10.1016/j.hpe.2020.02.002
ISSN: 2452-3011
eISSN: 2452-3011
Publisher: Elsevier B.V
Language: English
Date published: 06/2020
Academic Unit: Family and Community Medicine
Record Identifier: 9984297336602771

Metrics

32 Record Views