Journal article
Assessing the reproducibility of American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) arthroplasty studies
Journal of orthopaedic surgery and research, Vol.20(1), 216
03/01/2025
DOI: 10.1186/s13018-025-05538-0
PMCID: PMC11871755
PMID: 40022183
Abstract
Utilization of large-volume clinical registries for observational research has gained popularity in orthopaedic literature. However, concerns exist regarding inadequate reporting of methodology in this type of research. Despite these concerns, the reproducibility of such studies has not been adequately assessed in existing literature. This study aims to assess the reproducibility of American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) arthroplasty studies on smoking as a risk factor for poor surgical outcomes by employing identical datasets and statistical methods.
A systematic PubMed search between 2013 and 2023 identified ACS-NSQIP studies involving hip or knee arthroplasty and smoking as a potential risk factor for poor surgical outcomes. Each study's methods were reproduced by a trained statistician based on the reported methodology. In cases where certain steps were not explicitly stated, the statistician made informed decisions to reproduce those steps. Adjusted odds ratios (aORs) and p-values (α = 0.05) were compared between the original and reanalyzed datasets.
The initial search yielded 43 studies, with 11 meeting inclusion criteria resulting in the reanalysis of 268 aORs. Upon reanalysis, 12.69% of the original studies' aORs changed in interpretation, while 13.43% experienced a change in statistical significance. The average magnitude change of each aOR across all studies was 17.22%, and the sample size (N) in reanalysis varied by up to 47.84%. Among the 11 commonly cited studies, approximately one in eight objective conclusions changed in interpretation or statistical significance.
Inconsistent reproducibility exists across many arthroplasty studies that utilize the ACS-NSQIP database. These findings highlight the importance of rigorous reporting of study methodology, data collection, and statistical analyses when utilizing large-volume databases in orthopaedic research. This burden of responsibility should be shared among authors, peer reviewers, and orthopaedic journals to confirm the accuracy and validity of published database research.
This study systematically reviewed and analyzed, in attempt to reproduce, published arthroplasty studies utilizing ACS-NSQIP database to assess smoking as a potential risk factor for poor surgical outcomes. All analyzed studies included Level III Evidence, therefore this current study compares reproduced Level III Evidence to the original Level III Evidence.
Details
- Title: Subtitle
- Assessing the reproducibility of American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) arthroplasty studies
- Creators
- Ayobami S Ogunsola - University of IowaMichael C Marinier - University of IowaArman C Hlas - University of IowaJacob M Elkins - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Journal of orthopaedic surgery and research, Vol.20(1), 216
- DOI
- 10.1186/s13018-025-05538-0
- PMID
- 40022183
- PMCID
- PMC11871755
- NLM abbreviation
- J Orthop Surg Res
- ISSN
- 1749-799X
- eISSN
- 1749-799X
- Publisher
- BMC
- Language
- English
- Date published
- 03/01/2025
- Academic Unit
- Orthopedics and Rehabilitation
- Record Identifier
- 9984798357102771
Metrics
9 Record Views