Reliability of Outcome Monitoring Indices After Effort Exertion: A Psychometric Evaluation of the Effort-Doors Task for the Study of Individual Differences

Gavin Heindorf; Harold A Rocha; Tingting Chen; Walter P Vispoel; Peter E Clayson

doi:10.1111/psyp.70372

Back

Reliability of Outcome Monitoring Indices After Effort Exertion: A Psychometric Evaluation of the Effort-Doors Task for the Study of Individual Differences

Journal article

Open access

Peer reviewed

Reliability of Outcome Monitoring Indices After Effort Exertion: A Psychometric Evaluation of the Effort-Doors Task for the Study of Individual Differences

Gavin Heindorf, Harold A Rocha, Tingting Chen, Walter P Vispoel and Peter E Clayson

Psychophysiology, Vol.63(8), e70372

08/2026

DOI: 10.1111/psyp.70372

PMID: 42634175

Files and links (1)

url

https://doi.org/10.1111/psyp.70372View

Published (Version of record) Open Access

Abstract

The effort-doors task was developed to advance the study of individual differences in effort-reward dynamics by eliciting neural indices across multiple stages of outcome monitoring. The task's suitability for between-person inferences depends on the psychometric reliability of the scores it produces. Generalizability theory was used to evaluate the psychometric properties of the raw event-related brain potential (ERP) scores and their subtraction-based composites that are obtained from an extended version of the effort-doors task. An equation for estimating the reliability of a difference-of-differences score was derived and applied to estimate reliability and trial-count requirements at prespecified thresholds in a sample of 160 participants. The raw scores of cue-P3, reward positivity (RewP), and feedback-P3 (fb-P3) all achieved the recommended reliability threshold for between-person analyses. However, an effect of effort was observed only for cue-P3 (i.e., cue reactivity). The stimulus-preceding negativity (SPN) showed inadequate reliability; improved data quality or additional trials may increase reliability, although the projected trial requirement should be weighed against feasibility. Most raw scores from the 120-trial task were sufficiently reliable for between-person analyses. However, SPN reliability was inadequate, effort effects were not credible for SPN, RewP, or fb-P3, and fb-P3 did not vary by feedback valence. These limitations should be considered in relation to theory and alternative paradigms. Overall, the 120-trial version of the effort-doors task can support individual-differences research focused on raw ERP indices across most stages of outcome monitoring, whereas individual differences in effort-related, valence-related, or effort-by-valence modulation should not be inferred from subtraction-based scores without further optimization and demonstrated reliability.

Electroencephalography

Adolescent

Adult

Cues

Evoked Potentials - physiology

Female

Humans

Individuality

Male

Psychometrics - standards

Psychomotor Performance - physiology

Reproducibility of Results

Reward

Young Adult

Details

Title: Subtitle: Reliability of Outcome Monitoring Indices After Effort Exertion: A Psychometric Evaluation of the Effort-Doors Task for the Study of Individual Differences
Creators: Gavin Heindorf - University of South Florida
Harold A Rocha - University of South Florida
Tingting Chen - University of Iowa
Walter P Vispoel - University of Iowa
Peter E Clayson - University of South Florida
Resource Type: Journal article
Publication Details: Psychophysiology, Vol.63(8), e70372
DOI: 10.1111/psyp.70372
PMID: 42634175
NLM abbreviation: Psychophysiology
ISSN: 0048-5772
eISSN: 1469-8986
Publisher: Wiley
Grant note: R01MH128208 / NIMH NIH HHS
Language: English
Date published: 08/2026
Academic Unit: Psychological and Quantitative Foundations
Record Identifier: 9985220942302771

Metrics

3 Record Views