Journal article
Overall anesthesia department quality of clinical supervision of trainees over a year evaluated using mixed effects models
Journal of clinical anesthesia, Vol.87, 111114
08/01/2023
DOI: 10.1016/j.jclinane.2023.111114
PMID: 37004458
Abstract
Earlier studies of supervision in anesthesiology focused on how to evaluate the quality of individual anesthesiologist's clinical supervision of trainees. What is unknown is how to evaluate clinical supervision collectively, as provided by the department's faculty anesthesiologists. This information can be a metric that departments report annually or use to evaluate the effect of programs on the quality of clinical supervision over time.
This retrospective cohort study used all 48,788 evaluations of the 115 faculty anesthesiologists using the De Oliveira Filho supervision scale completed by 202 residents and fellows over nine academic years at one department.
The distributions of mean scores among raters had marked negative skewness and were inconsistent with normal distributions. Consequently, accurate confidence intervals were impracticably wide, and their interpretation suggested lack of validity. In contrast, the logits of the proportions of scores equaling the maximum possible value, calculated for each rater, followed distributions sufficiently close to normal for statistically reliable use in random effects modeling. Parameters and confidence intervals were estimated using the intercept only random effects models, and then inverses computed to convert results from the logit scale to proportions. That approach is analogous to random effect meta-analysis of proportional incidence (or prevalence). Departments that chose to use semi-annual or annual surveys of trainees regarding supervision quality, and report those raw counts, will have far lower estimates of supervision quality versus when calculated accurately using daily evaluations of individual anesthesiologists.
Random effects meta-analysis of percentage incidences of maximum scores is a suitable statistical approach to analyze the daily supervision scores of individual anesthesiologists to evaluate the overall quality of clinical supervision provided to the trainees by the department over a year.
•Goal is to evaluate overall anesthesia departmental clinical supervision annually.•Retrospective cohort study performed, with 48,788 evaluations by 202 residents and fellows.•Mean scores markedly negatively skewed making accurate confidence intervals very wide.•Random effects meta-analysis of proportional incidence of quality supervision can be used.•Random effects meta-analysis weights each rater individually, decreasing institutional bias.
Details
- Title: Subtitle
- Overall anesthesia department quality of clinical supervision of trainees over a year evaluated using mixed effects models
- Creators
- Franklin Dexter - Division of Management Consulting, Department of Anesthesia, University of Iowa, United States of AmericaBradley J. Hindman - Department of Anesthesia, University of Iowa, United States of AmericaRichard H. Epstein - University of Miami
- Resource Type
- Journal article
- Publication Details
- Journal of clinical anesthesia, Vol.87, 111114
- Publisher
- Elsevier Inc
- DOI
- 10.1016/j.jclinane.2023.111114
- PMID
- 37004458
- ISSN
- 0952-8180
- eISSN
- 1873-4529
- Language
- English
- Date published
- 08/01/2023
- Academic Unit
- Health Management and Policy; Anesthesia
- Record Identifier
- 9984386252702771
Metrics
14 Record Views