Journal article
Examiner error in curriculum-based measurement of oral reading
Journal of school psychology, Vol.52(4), pp.361-375
08/2014
DOI: 10.1016/j.jsp.2014.05.007
PMID: 25107409
Abstract
Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.
Details
- Title: Subtitle
- Examiner error in curriculum-based measurement of oral reading
- Creators
- Kelli D Cummings - Center on Teaching and Learning, University of Oregon, USAGina Biancarosa - Educational Methodology, Policy, and Leadership, University of Oregon, USAAndrew Schaper - Educational Methodology, Policy, and Leadership, University of Oregon, USADeborah K Reed - Florida Center for Reading Research, Florida State University, USA
- Resource Type
- Journal article
- Publication Details
- Journal of school psychology, Vol.52(4), pp.361-375
- Publisher
- Elsevier Ltd
- DOI
- 10.1016/j.jsp.2014.05.007
- PMID
- 25107409
- ISSN
- 0022-4405
- eISSN
- 1873-3506
- Grant note
- Center on Teaching and Learning (CTL)
- Language
- English
- Date published
- 08/2014
- Academic Unit
- Teaching and Learning
- Record Identifier
- 9983993486902771
Metrics
21 Record Views