Journal article
The reliability of summative judgements based on objective structured clinical examination cases distributed across the clinical year
Medical education, Vol.41(7), pp.661-666
07/2007
DOI: 10.1111/j.1365-2923.2007.02786.x
PMID: 17614886
Abstract
Objective structured clinical examinations (OSCEs) can be used for formative and summative evaluation. We sought to determine the generalisability of students' summary scores aggregated from formative OSCE cases distributed across 5 clerkships during Year 3 of medical school.
Five major clerkships held OSCEs with 2-4 cases each during their rotations. All cases used 15-minute student-standardised patient encounters and performance was assessed using clinical and communication skills checklists. As not all students completed every clerkship or OSCE case, the generalisability (G) study was an unbalanced student x (case : clerkship) design. After completion of the G study, a decision (D) study was undertaken and phi (phi) values for different cut-points were calculated.
The data for this report were collected over 2 academic years involving 262 Year 3 students. The G study found that 9.7% of the score variance originated from the student, 3.1% from the student-clerkship interaction, and 87.2% from the student-case nested within clerkship effect. Using the variance components from the G study, the D study suggested that if students completed 3 OSCE cases in each of the 5 different clerkships, the reliability of the aggregated scores would be 0.63. The phi, calculated at a cut-point 1 standard deviation below the mean, would be approximately 0.85.
Aggregating case scores from low stakes OSCEs within clerkships results in a score set that allows for very reliable decisions about which students are performing poorly. Medical schools can use OSCE case scores collected over a clinical year for summative evaluation.
Details
- Title: Subtitle
- The reliability of summative judgements based on objective structured clinical examination cases distributed across the clinical year
- Creators
- George R Bergus - Department of Family Medicine, Carver College of Medicine, University of Iowa, Iowa City, Iowa 52245, USA. george-bergus@uiowa.eduClarence D Kreiter
- Resource Type
- Journal article
- Publication Details
- Medical education, Vol.41(7), pp.661-666
- Publisher
- England
- DOI
- 10.1111/j.1365-2923.2007.02786.x
- PMID
- 17614886
- ISSN
- 0308-0110
- eISSN
- 1365-2923
- Language
- English
- Date published
- 07/2007
- Academic Unit
- Physician Assistant Studies; Psychiatry; Family and Community Medicine
- Record Identifier
- 9984006419602771
Metrics
28 Record Views