Journal article
Beyond classical metrics: Generalizability theory across psychophysiological modalities
International journal of psychophysiology, 113321
01/09/2026
DOI: 10.1016/j.ijpsycho.2026.113321
PMID: 41520835
Abstract
Psychophysiological research relies on biological measures to understand cognitive, affective, and behavioral processes, but the utility of these measures for studying individual differences depends on their psychometric reliability. Traditional reliability methods, such as classical test theory, often fail to account for the multiple sources of variance inherent in psychophysiological data. Generalizability theory (GT) provides a robust, multifaceted approach to reliability estimation by decomposing variance across multiple facets, such as trials, tasks, and sessions. This article introduces GT to psychophysiological researchers, detailing its advantages over classical approaches and demonstrating its application to a variety of psychophysiological modalities: event-related potentials (ERPs), electroencephalography (EEG), electrodermal activity (EDA), electromyography (EMG), and electrocardiography (ECG). We outline the two-phase process of GT: generalizability (G) studies, which quantify variance components, and decision (D) studies, which optimize reliability within study designs intended for specific purposes. Psychometric formulas are provided for estimating indices of generalizability, dependability, and measurement error for numerous designs, including ones based on difference scores. Additionally, we discuss best practices for variance component estimation, highlighting the advantages of multilevel modeling in handling unbalanced data and non-normal distributions, typical of psychophysiological data. By applying GT, researchers can enhance the replicability and interpretability of psychophysiological measures, ultimately strengthening their ability to link biological signals to psychological constructs. This framework represents a necessary evolution in psychophysiological science, ensuring that biological measurements are grounded in fundamental psychometric principles.
•Generalizability theory evaluates reliability of biological measurements•Applies framework to EEG, ERPs, EDA, EMG, and EDA•Provides formulas for reliability appropriate for biological measurements•Guides trial and session planning to meet explicit reliability thresholds
Details
- Title: Subtitle
- Beyond classical metrics: Generalizability theory across psychophysiological modalities
- Creators
- Harold A. Rocha - University of South FloridaAmanda Holbrook - University of South FloridaGreg Hajcak - Santa Clara UniversityAndreas Keil - University of FloridaPhilippe Rast - University of California, DavisJulian F. Thayer - Irvine UniversityEdelyn Verona - University of South FloridaWalter P. Vispoel - University of IowaPeter E. Clayson - University of South Florida
- Resource Type
- Journal article
- Publication Details
- International journal of psychophysiology, 113321
- DOI
- 10.1016/j.ijpsycho.2026.113321
- PMID
- 41520835
- NLM abbreviation
- Int J Psychophysiol
- ISSN
- 0167-8760
- eISSN
- 1872-7697
- Publisher
- Elsevier B.V
- Language
- English
- Electronic publication date
- 01/09/2026
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985121593702771
Metrics
3 Record Views