Logo image
Back
The validity of orthopaedic in-training examination scores
Journal article   Peer reviewed

The validity of orthopaedic in-training examination scores

Joseph A Buckwalter, Rex Schumacher, John P Albright and Reginald R Cooper
Journal of bone and joint surgery. American volume, Vol.63(6), pp.1001-1006
07/1981
DOI: 10.2106/00004623-198163060-00018
PMID: 7240314

View Online

Abstract

The value of the Orthopaedic In-Training Examination (OITE) depends on its validity in measuring orthopaedic knowledge. To assess the concurrent validity of the OITE, we asked three staff members in each of twenty orthopaedic residency programs to rank their residents on the basis or performance in orthopaedics. In each of sixteen programs, three staff members assigned rank orders to all of their residents, but in four programs either less than three staff orthopaedists ranked their residents or their rankings were not completed according to instructions. Inter-rater reliability analyses showed that the staff members in each program generally agreed closely in their rankings of their residents (mean Cronbach's α = 0.83; mean Kendall's coefficient of concordance, W = 0.76). We then compared the resident's mean performance ranks with their OITE scores. Over-all, the mean performance ranks correlated positively with the OITE scores (mean r = 0.53), but the relationship varied considerably between programs from a high value of r = 0.85 in two programs to a low value of r = 0.12 in one program. Both the OITE scores and the mean performance ranks commonly increased with more years of training. However, a positive correlation still existed between the OITE scores and the mean performance ranks when the years of training were held constant statistically. These results indicate that staff members familiar with residents' work can reproducibly rank a resident's performance relative to that of other residents in their program, that the OITE measures knowledge as related to resident performance in orthopaedics, and that OITE scores may be useful to residents in assessing their performance relative to that of other residents. The strength of the relationships between OITE scores and performance rank varies between residency programs, and this relationship must be interpreted independently for each individual program.
 Internship and Residency Educational Measurement Orthopedics - education

Details

Metrics

22 Record Views
20 Times Cited - Web of Science
Logo image