Journal article
The Impact of Using Mean Versus Mode When Assessing Resident Competency
Journal of graduate medical education, Vol.9(3), pp.302-309
06/2017
DOI: 10.4300/JGME-D-16-00571.1
PMCID: PMC5476378
PMID: 28638507
Abstract
Background
The Accreditation Council for Graduate Medical Education Milestone Project was implemented in 2014 to standardize assessments and progression of residents. While it is recommended that milestones not be used as tools for direct assessments of resident competency, many programs have used or adapted milestone tools for this purpose.
Objective
We sought to explore use of the most frequent milestone level at which a resident was evaluated (ie, the mode), and compared this to the standard practice of using the arithmetic mean for summarizing performance.
Methods
We reviewed all Family Medicine Milestone evaluations from 1 program for the first 2 academic years of milestone implementation. Mean and mode scores were calculated across 24 unique residents, 841 evaluation forms, and 5897 measurements. The proportion of overestimation errors (where the mean is at least 0.5 larger than the mode) and underestimation errors (where the mean is at least 0.5 less than the mode) were then compared across resident training year and subcompetency.
Results
For the 24 residents, an estimation error occurred in 175 of 792 of the comparisons (22%). Of these errors, 118 (67%) were overestimation errors. First-year residents accounted for 55% (96 of 175) of all estimation errors. All subcompetencies had some estimation errors, with 6 having greater than 5%.
Conclusions
If the trend for using the milestones as stand-alone assessment tools is to continue, aggregating data by using frequency distributions and mode would be a more stable and appropriate approach given their nominal or, at best, ordinal nature.
Details
- Title: Subtitle
- The Impact of Using Mean Versus Mode When Assessing Resident Competency
- Creators
- Patrick B Barlow - University of Iowa, General Internal MedicineKate DuChene Thoma - University of Iowa, Family and Community MedicineKristi J Ferguson - University of Iowa, Internal Medicine
- Resource Type
- Journal article
- Publication Details
- Journal of graduate medical education, Vol.9(3), pp.302-309
- DOI
- 10.4300/JGME-D-16-00571.1
- PMID
- 28638507
- PMCID
- PMC5476378
- NLM abbreviation
- J Grad Med Educ
- ISSN
- 1949-8349
- eISSN
- 1949-8357
- Publisher
- The Accreditation Council for Graduate Medical Education
- Language
- English
- Date published
- 06/2017
- Academic Unit
- ICTS; Family and Community Medicine; Center for Social Science Innovation; Office of Consultation and Research in Medical Education; General Internal Medicine; Internal Medicine
- Record Identifier
- 9984203348302771
Metrics
16 Record Views