Report
Expert/Novice Differences in the Focus and Procedures Used by Essay Scorers
Distributed by ERIC Clearinghouse
04/1996
Abstract
The amount of variability contributed to large-scale performance assessment scores by raters is a constant concern for those who wish to use results from these assessments for educational decisions. This study approaches the problem by examining the behaviors of essay scorers who demonstrate different levels of proficiency with a holistic scoring rubric. Scorers (n=36) for a large-scale writing assessment scoring project performed a think-aloud task as they scored 24 essays. The protocols were analyzed by examining differences in the ways expert, intermediate, and novice scorers processed each essay and the content focused on in each essay. Two conclusions are drawn, based on these analyses. The first is that expert scorers are more likely to use more fluent processing methods to score essays than are nonexperts. The second is that there is little evidence to suggest that the content that is focused upon by essay scorers is related to scoring proficiency. (Contains 2 figures, 6 tables, and 17 references.) (Author)
Details
- Title: Subtitle
- Expert/Novice Differences in the Focus and Procedures Used by Essay Scorers
- Creators
- Edward W WolfeChi-Wen Kao
- Resource Type
- Report
- Publisher
- Distributed by ERIC Clearinghouse
- Number of pages
- 31 pages
- Language
- English
- Date published
- 04/1996
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985134745302771
Metrics
1 Record Views