Journal article
Measuring to Improve: Peer and Crowd-sourced Assessments of Technical Skill with Robot-assisted Radical Prostatectomy
European urology, Vol.69(4), pp.547-550
04/01/2016
DOI: 10.1016/j.eururo.2015.11.028
PMID: 26755338
Abstract
Because surgical skill may be a key determinant of patient outcomes, there is growing interest in skill assessment. In the Michigan Urological Surgery Improvement Collaborative (MUSIC), we assessed whether peer and crowd-sourced (ie, layperson) video review of robot-assisted radical prostatectomy (RARP) could distinguish technical skill among practicing surgeons. A total of 76 video clips from 12 MUSIC surgeons consisted of one of four parts of RARP and underwent blinded review by MUSIC peer surgeons and prequalified crowd-sourced reviewers. Videos were rated for global skill (Global Evaluation Assessment of Robotic Skills) and procedure-specific skill (Robotic Anastomosis and Competency Evaluation). We fit linear mixed-effects models to estimate mean peer and crowd ratings for each video. Individual video ratings were aggregated to calculate surgeon skill scores. Peers (n=25) completed 351 video ratings over 15 d, whereas crowd-sourced reviewers (n=680) completed 2990 video ratings in 38h. Surgeon global skill scores ranged from 15.8 to 21.7 (peer) and from 19.2 to 20.9 (crowd). Peer and crowd ratings demonstrated strong correlation for both global (r=0.78) and anastomosis (r=0.74) skills. The two groups consistently agreed on the rank order of lower scoring surgeons, suggesting a potential role for crowd-sourced methodology in the assessment of surgical performance. Lack of patient outcomes is a limitation and forms the basis of future study.
We demonstrated the large-scale feasibility of assessing the technical skill of robotic surgeons and found that online crowd-sourced reviewers agreed with experts on the rank order of surgeons with the lowest technical skill scores.
A Web-based video review of the technical skill of surgeons perfoming robotic prostatectomy is feasible within a structured framework. Assessments by crowd-sourced (ie, nonexpert) and peer surgeon reviewers correlated closely, especially for videos with lower skill ratings.
Details
- Title: Subtitle
- Measuring to Improve: Peer and Crowd-sourced Assessments of Technical Skill with Robot-assisted Radical Prostatectomy
- Creators
- Khurshid R. Ghani - University of MichiganDavid C. Miller - University of MichiganSusan Linsell - University of MichiganAndrew Brachulis - University of MichiganBrian Lane - Spectrum HealthRichard Sarle - Michigan Institute of UrologyDeepansh Dalela - Henry Ford Health SystemMani Menon - Henry Ford Health SystemBryan Comstock - University of WashingtonThomas S. Lendvay - University of WashingtonJames Montie - University of MichiganJames O. Peabody - Henry Ford Health SystemMichigan Urological Surgery Improvement Collaborative
- Resource Type
- Journal article
- Publication Details
- European urology, Vol.69(4), pp.547-550
- DOI
- 10.1016/j.eururo.2015.11.028
- PMID
- 26755338
- NLM abbreviation
- Eur Urol
- ISSN
- 0302-2838
- eISSN
- 1873-7560
- Publisher
- Elsevier B.V
- Number of pages
- 4
- Language
- English
- Date published
- 04/01/2016
- Academic Unit
- Urology
- Record Identifier
- 9984949451402771
Metrics
1 Record Views