Journal article
A meta-analytic perspective on the valid use of subjective human judgement to make medical school admission decisions
Medical education online, Vol.23(1), pp.1522225-9
10/05/2018
DOI: 10.1080/10872981.2018.1522225
PMCID: PMC6179055
PMID: 30286694
Abstract
While medical educators appear to believe that admission to the medical school should be governed, at least in part, by human judgement, there has been no systematic presentation of evidence suggesting it improves selection. From a fair testing perspective, legal, ethical, and psychometric considerations, all dictate that the scientific evidence regarding human judgement in selection should be given consideration. To investigate the validity of using human judgements in admissions, multi-disciplinary meta-analytic research evidence from the wider literature is combined with studies from within medical education to provide evidence regarding the fairness and validity of using interviews and holistic review in medical school admissions. Fourteen studies, 6 of which are meta-analytic studies that summarized 292 individual studies, were included in the final review. Within these studies, a total of 33 studies evaluated the reliability of the traditional interview. These studies reveal that the interview has low to moderate reliability (similar to.42) which significantly limits its validity. This is confirmed by over 100 studies examining interview validity which collectively show interview scores to be moderately correlated with important outcome variables (corrected value similar to.29). Meta-analyses of over 150 studies demonstrate that mechanical/formula-based selection decisions produce better results than decisions made with holistic/clinical methods (human judgement). Three conclusions regarding the use of interviews and holistic review are provided by these meta-analyses. First, it is clear that the traditional interview has low reliability and that this significantly limits its validity. Second, the reliable variance from interview scores appears moderately predictive of outcomes that are relevant to consider in medical school admission. And third, the use of holistic review as a method of incorporating human judgement is not a valid alternative to mechanical/statistical approaches as the evidence clearly indicates that mechanistic methods are more predictive, reliable, cost efficient, and transparent.
Details
- Title: Subtitle
- A meta-analytic perspective on the valid use of subjective human judgement to make medical school admission decisions
- Creators
- Clare Kreiter - Roy J. and Lucille A. Carver College of MedicineMarie O'Shea - Royal College of Surgeons in IrelandCatherine Bruen - Royal College of Surgeons in IrelandPaul Murphy - Royal College of Surgeons in IrelandTeresa Pawlikowska - Royal College of Surgeons in Ireland
- Resource Type
- Journal article
- Publication Details
- Medical education online, Vol.23(1), pp.1522225-9
- DOI
- 10.1080/10872981.2018.1522225
- PMID
- 30286694
- PMCID
- PMC6179055
- NLM abbreviation
- Med Educ Online
- ISSN
- 1087-2981
- eISSN
- 1087-2981
- Publisher
- Taylor & Francis
- Number of pages
- 9
- Grant note
- name: Fulbright Scholar grant; DOI: 10.13039/100012921, name: Royal College of Surgeons in Ireland
- Language
- English
- Date published
- 10/05/2018
- Academic Unit
- Family and Community Medicine
- Record Identifier
- 9984297445002771
Metrics
23 Record Views