Journal article
Rater Model Using Signal Detection Theory for Latent Differential Rater Functioning
Multivariate behavioral research, Vol.54(4), pp.492-504
07/04/2019
DOI: 10.1080/00273171.2018.1522496
PMID: 30557061
Abstract
Differential rater functioning (DRF) occurs when raters show evidence of exercising differential severity or leniency when scoring examinees within different subgroups. Previous studies of DRF have examined rater bias using manifest variables (e.g., use of covariates) to determine the subgroups. These manifest variables include gender and the ethnicity of the examinee. For example, a rater may score males more severely. Ideally, each rater's severity should be invariant across subgroups. This study examines DRF in the context of latent subgroups that classify possible sources of DRF based on raters' scoring behavior rather than manifest factors. An extension of the latent class signal detection theory (LC-SDT) model for identifying DRF is proposed and examined using real-world data and simulations. Results from real-world data show that the signal detection approach leads to an effective method to identify latent DRF. Simulations with varying sample sizes and conditions of rater precision were shown to recover parameters at an adequate level, supporting its use to identify latent DRF in large-scale data. These findings suggest that the DRF extension of the LC-SDT can be a useful model to examine characteristics of raters and add information that can aid rater training.
Details
- Title: Subtitle
- Rater Model Using Signal Detection Theory for Latent Differential Rater Functioning
- Creators
- Yoon Soo Park - University of Illinois at ChicagoKuan Xing - University of Illinois at Chicago
- Resource Type
- Journal article
- Publication Details
- Multivariate behavioral research, Vol.54(4), pp.492-504
- Publisher
- Routledge
- DOI
- 10.1080/00273171.2018.1522496
- PMID
- 30557061
- ISSN
- 0027-3171
- eISSN
- 1532-7906
- Language
- English
- Date published
- 07/04/2019
- Academic Unit
- Family and Community Medicine; Office of Consultation and Research in Medical Education
- Record Identifier
- 9984658249902771
Metrics
4 Record Views