Journal article
Determination of Interrater Reliability of a Universal Evaluator Rubric to Assess Student Pharmacist Communication Skills
American journal of pharmaceutical education, Vol.86(1), 8544
01/2022
DOI: 10.5688/ajpe8544
PMCID: PMC8787169
PMID: 34301546
Abstract
To evaluate the interrater reliability of a universal evaluator rubric used to assess student pharmacist communication skills during patient education sessions.
Six US schools and colleges of pharmacy each submitted 10 student videos of a simulated community pharmacy patient education session and recruited two raters in each of the five rater groups (faculty, standardized patients, postgraduate year one residents, student pharmacists, and pharmacy preceptors). Raters used a rubric containing 20 items and a global assessment to evaluate student communication of 12 videos. Agreement was computed for individual items and overall rubric score within each rater group, and for each item across all rater groups. Average overall rubric agreement scores were compared between rater groups. Agreement coefficient scores were categorized as no to minimal, weak, moderate, strong, or almost perfect agreement.
Fifty-five raters representing five rater groups and six pharmacy schools evaluated student communication. Item agreement analysis for all raters revealed five items with no to minimal or weak agreement, 10 items with moderate agreement, one item with strong agreement, and five items with almost perfect agreement. Overall average agreement across all rater groups was 0.73 (95% CI, 0.66-0.81). The preceptor rater group exhibited the lowest agreement score of 0.68 (95% CI, 0.58-0.78), which significantly deviated from the overall average.
While strong or almost perfect agreement scores were not observed for all rubric items, overall average interrater reliability results support the use of this rubric in a variety of raters to assess student pharmacist communication skills during patient education sessions.
Details
- Title: Subtitle
- Determination of Interrater Reliability of a Universal Evaluator Rubric to Assess Student Pharmacist Communication Skills
- Creators
- Susanne G Barnett - University of Wisconsin–MadisonSheila M Allen - University of Illinois ChicagoKaren Ms Bastianelli - University of MinnesotaJennifer S Chen - University of MinnesotaColleen A Clark Dula - The Ohio State UniversityMarlowe Djuric Kachlic - University of Illinois ChicagoKristen L Goliak - University of Illinois ChicagoLaura E Knockel - University of IowaDavid E Matthews - The Ohio State UniversityLucio R Volino - Rutgers, The State University of New JerseyMichael R Lasarev - University of Wisconsin–MadisonJeffrey C Reist - University of Iowa
- Resource Type
- Journal article
- Publication Details
- American journal of pharmaceutical education, Vol.86(1), 8544
- DOI
- 10.5688/ajpe8544
- PMID
- 34301546
- PMCID
- PMC8787169
- NLM abbreviation
- Am J Pharm Educ
- ISSN
- 0002-9459
- eISSN
- 1553-6467
- Language
- English
- Date published
- 01/2022
- Academic Unit
- Pharmacy Practice and Science; Injury Prevention Research Center
- Record Identifier
- 9984365894002771
Metrics
57 Record Views