Journal article
Bernoulli Cumulative Sum (CUSUM) control charts for monitoring of anesthesiologists' performance in supervising anesthesia residents and nurse anesthetists
Anesthesia and analgesia, Vol.119(3), pp.679-685
09/2014
DOI: 10.1213/ANE.0000000000000342
PMID: 25137002
Abstract
We describe our experiences in using Bernoulli cumulative sum (CUSUM) control charts for monitoring clinician performance. The supervision provided by each anesthesiologist is evaluated daily by the Certified Registered Nurse Anesthetists (CRNAs) and/or anesthesia residents with whom they work. Each of 9 items is evaluated (1 = never, 2 = rarely, 3 = frequently, 4 = always). The score is the mean of the 9 responses. Choosing thresholds for low scores is straightforward, <2.0 for CRNAs and <3.0 for residents. Bernoulli CUSUM detection of low scores was within 50 ± 14 (median ± quartile deviation) days rather than 182 days without use of CUSUM. The true positive detection of anesthesiologists with incidences of low scores greater than the chosen "out-of-control" rate was 14 of 14. The false-positive detection rate was 0 of 29. This CUSUM performance exceeded that of Shewhart individual control charts, for which the smallest threshold sufficiently large to detect 14 of 14 true positives had false-positive detection of 16 of 29 anesthesiologists. The Bernoulli CUSUM assumes that scores are known right away, which is untrue. However, CUSUM performance was insensitive to this assumption. The Bernoulli CUSUM assumes statistical independence of scores, which also is untrue. For example, when an evaluation of an anesthesiologist 1 day by a CRNA had a low score, there was an increased chance that another CRNA working in a different operating room on the same day would also give that same anesthesiologist a low score (P < 0.0001). This correlation among scores does affect the Bernoulli CUSUM, such that detection is more likely. This is an advantage for our continual process improvement application since it flags individuals for further evaluation by managers while maintaining confidentiality of raters.
Details
- Title: Subtitle
- Bernoulli Cumulative Sum (CUSUM) control charts for monitoring of anesthesiologists' performance in supervising anesthesia residents and nurse anesthetists
- Creators
- Franklin Dexter - From the Division of Management Consulting, Department of Anesthesia, †Department of Management Sciences, and ‡Department of Anesthesia, University of Iowa, Iowa City, IowaJohannes LedolterBradley J Hindman
- Resource Type
- Journal article
- Publication Details
- Anesthesia and analgesia, Vol.119(3), pp.679-685
- Publisher
- United States
- DOI
- 10.1213/ANE.0000000000000342
- PMID
- 25137002
- ISSN
- 0003-2999
- eISSN
- 1526-7598
- Language
- English
- Date published
- 09/2014
- Academic Unit
- Statistics and Actuarial Science; Health Management and Policy; Anesthesia; Business Analytics
- Record Identifier
- 9983806380202771
Metrics
31 Record Views