Logo image
Back
Reproducibility in the absence of selective reporting: An illustration from large-scale brain asymmetry research
Journal article   Open access   Peer reviewed

Reproducibility in the absence of selective reporting: An illustration from large-scale brain asymmetry research

Xiang-Zhen Kong, Clyde Francks and ENIGMA Laterality Working Group
Human brain mapping, Vol.43(1), pp.244-254
01/2022
DOI: 10.1002/hbm.25154
PMCID: PMC8675427
PMID: 32841457
url
https://doi.org/10.1002/hbm.25154View
Published (Version of record) Open Access

Abstract

The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p-hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left-right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta-analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an "ideal publishing environment," that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically-used sample sizes.
 Adolescent Adult Aged Brain Cortical Thickness Cerebral Cortex - anatomy & histology Cerebral Cortex - diagnostic imaging Datasets as Topic Humans Magnetic Resonance Imaging - standards Middle Aged Multicenter Studies as Topic - standards Neuroimaging - standards Publication Bias Reproducibility of Results Young Adult

Details

Metrics

15 Record Views
16 Times Cited - Web of Science
Logo image