Logo image
Back
Case Sampling vs Universal Review for Evaluating Hospital Postoperative Mortality in US Surgical Quality Improvement Programs
Journal article   Peer reviewed

Case Sampling vs Universal Review for Evaluating Hospital Postoperative Mortality in US Surgical Quality Improvement Programs

Vivi W. Chen, Alexis P. Chidi, Tracey Rosen, Yongquan Dong, Peter A. Richardson, Jennifer Kramer, David A. Axelrod, Laura A. Petersen and Nader N. Massarweh
JAMA surgery, Vol.158(12), pp.1312-1319
12/01/2023
DOI: 10.1001/jamasurg.2023.4532
PMCID: PMC10535011
PMID: 37755869
url
https://www.ncbi.nlm.nih.gov/pmc/articles/10535011View
Open Access

Abstract

Importance Representative surgical case sampling, rather than universal review, is used by US Department of Veterans Affairs (VA) and private-sector national surgical quality improvement (QI) programs to assess program performance and to inform local QI and performance improvement efforts. However, it is unclear whether case sampling is robust for identifying hospitals with safety or quality concerns. Objective To evaluate whether the sampling strategy used by several national surgical QI programs provides hospitals with data that are representative of their overall quality and safety, as measured by 30-day mortality. Design, Setting, and Participants This comparative effectiveness study was a national, hospital-level analysis of data from adult patients (aged ≥18 years) who underwent noncardiac surgery at a VA hospital between January 1, 2016, and September 30, 2020. Data were obtained from the VA Surgical Quality Improvement Program (representative sample) and the VA Corporate Data Warehouse surgical domain (100% of surgical cases). Data analysis was performed from July 1 to December 21, 2022. Main Outcomes and Measures The primary outcome was postoperative 30-day mortality. Quarterly, risk-adjusted, 30-day mortality observed-to-expected (O-E) ratios were calculated separately for each hospital using the sample and universal review cohorts. Outlier hospitals (ie, those with higher-than-expected mortality) were identified using an O-E ratio significantly greater than 1.0. Results In this study of data from 113 US Department of Veterans Affairs hospitals, the sample cohort comprised 502 953 surgical cases and the universal review cohort comprised 1 703 140. The majority of patients in both the representative sample and the universal sample were men (90.2% vs 91.1%) and were White (74.7% vs 74.5%). Overall, 30-day mortality was 0.8% and 0.6% for the sample and universal review cohorts, respectively ( P < .001). Over 2145 quarters of data, hospitals were identified as an outlier in 11.7% of quarters with sampling and in 13.2% with universal review. Average hospital quarterly 30-day mortality rates were 0.4%, 0.8%, and 0.9% for outlier hospitals identified using the sample only, universal review only, and concurrent identification in both data sources, respectively. For nonsampled cases, average hospital quarterly 30-day mortality rates were 1.0% at outlier hospitals and 0.5% at nonoutliers. Among outlier hospital quarters in the sample, 47.4% were concurrently identified with universal review. For those identified with universal review, 42.1% were concurrently identified using the sample. Conclusions and Relevance In this national, hospital-level study, sampling strategies employed by national surgical QI programs identified less than half of hospitals with higher-than-expected perioperative mortality. These findings suggest that sampling may not adequately represent overall surgical program performance or provide stakeholders with the data necessary to inform QI efforts.

Details

Metrics

16 Record Views
9 Times Cited - Web of Science
Logo image