Journal article
Beyond classical metrics: Generalizability theory across psychophysiological modalities
International journal of psychophysiology, Vol.222, 113321
04/2026
DOI: 10.1016/j.ijpsycho.2026.113321
PMID: 41520835
Abstract
Psychophysiological research relies on biological measures to understand cognitive, affective, and behavioral processes, but the utility of these measures for studying individual differences depends on their psychometric reliability. Traditional reliability methods, such as classical test theory, often fail to account for the multiple sources of variance inherent in psychophysiological data. Generalizability theory (GT) provides a robust, multifaceted approach to reliability estimation by decomposing variance across multiple facets, such as trials, tasks, and sessions. This article introduces GT to psychophysiological researchers, detailing its advantages over classical approaches and demonstrating its application to a variety of psychophysiological modalities: event-related potentials (ERPs), electroencephalography (EEG), electrodermal activity (EDA), electromyography (EMG), and electrocardiography (ECG). We outline the two-phase process of GT: generalizability (G) studies, which quantify variance components, and decision (D) studies, which optimize reliability within study designs intended for specific purposes. Psychometric formulas are provided for estimating indices of generalizability, dependability, and measurement error for numerous designs, including ones based on difference scores. Additionally, we discuss best practices for variance component estimation, highlighting the advantages of multilevel modeling in handling unbalanced data and non-normal distributions, typical of psychophysiological data. By applying GT, researchers can enhance the replicability and interpretability of psychophysiological measures, ultimately strengthening their ability to link biological signals to psychological constructs. This framework represents a necessary evolution in psychophysiological science, ensuring that biological measurements are grounded in fundamental psychometric principles.
•Generalizability theory evaluates reliability of biological measurements•Applies framework to EEG, ERPs, EDA, EMG, and EDA•Provides formulas for reliability appropriate for biological measurements•Guides trial and session planning to meet explicit reliability thresholds
Details
- Title: Subtitle
- Beyond classical metrics: Generalizability theory across psychophysiological modalities
- Creators
- Harold A. Rocha - University of South FloridaAmanda Holbrook - University of South FloridaGreg Hajcak - Santa Clara UniversityAndreas Keil - University of FloridaPhilippe Rast - University of California, DavisJulian F. Thayer - Irvine UniversityEdelyn Verona - University of South FloridaWalter P. Vispoel - University of IowaPeter E. Clayson - University of South Florida
- Resource Type
- Journal article
- Publication Details
- International journal of psychophysiology, Vol.222, 113321
- DOI
- 10.1016/j.ijpsycho.2026.113321
- PMID
- 41520835
- NLM abbreviation
- Int J Psychophysiol
- ISSN
- 0167-8760
- eISSN
- 1872-7697
- Publisher
- Elsevier B.V
- Grant note
- NIMH: MH128208
The present article was supported by an NIMH grant awarded to Peter Clayson (MH128208) . This funding source was not involved in data analysis, interpretation, or the preparation of this submission for publication. Peter Clayson serves as an associate editor at the Interna-tional Journal of Psychophysiology and had no involvement in the peer review or editorial decision making for this article. Greg Hajcak has received consulting fees from Universal Brain, LLC and Alto Neuroscience, Inc. During the preparation of this work, the authors used GPT-5.2 through ChatGPT to assist in streamlining and proofreading text. After using ChatGPT, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. We thank Bradley Buchanan, Tingting Chen, Hyeryung Lee, and John Shuford for comments provided on earlier drafts of the article.
- Language
- English
- Electronic publication date
- 01/09/2026
- Date published
- 04/2026
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985121593702771
Metrics
3 Record Views