Journal article
The relationship between essay reading style and scoring proficiency in a psychometric scoring system
Assessing writing, Vol.4(1), pp.83-106
1997
DOI: 10.1016/S1075-2935(97)80006-2
Abstract
This article presents a model that describes the cognitive frameworks and processes used by essay scorers in a large scale psychometric scoring system. Based on this model, an investigation of the relationship between a scorer's reading style and proficiency with a scoring rubric is reported. Scorers who demonstrated different levels of proficiency (i.e., different levels of interrater agreement) with a narrative scoring rubric performed a think aloud task. Protocols were analyzed to determine whether scoring proficiency is manifested in the way a scorer considers various features of the essay, makes judgments about the quality of the essay, and makes personal comments about the essay and scoring task. The results suggest that proficient scorers are better able to withhold judgment and may focus their efforts more intensely on the process of evaluation. Implications of these results for selecting, evaluating, and training scorers are discussed.
Details
- Title: Subtitle
- The relationship between essay reading style and scoring proficiency in a psychometric scoring system
- Creators
- Edward W. Wolfe - University of Florida
- Resource Type
- Journal article
- Publication Details
- Assessing writing, Vol.4(1), pp.83-106
- DOI
- 10.1016/S1075-2935(97)80006-2
- ISSN
- 1075-2935
- eISSN
- 1873-5916
- Publisher
- Elsevier Inc
- Number of pages
- 24
- Language
- English
- Date published
- 1997
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985123694702771
Metrics
1 Record Views