Journal article
Integrated writing scores based on holistic and multi-trait scales: A generalizability analysis
Assessing writing, Vol.38, pp.21-36
10/01/2018
DOI: 10.1016/j.asw.2018.08.001
Abstract
Scoring reading-to-write (RTW) tasks is known to be more challenging than independent tasks given that raters should attend to features of source use, in addition to other writing skills. Holistic scales have been traditionally used with this task type; however, analytic scales have recently received increasing attention. While research has looked into score generalizability of RTW tasks, few studies have addressed the impact of rating scales on RTW score reliability. For this purpose, the current study compares score reliability from both holistic and multi-trait rating scales. Following a generalizability theory approach, five raters scored 60 essays written by EFL university students using both holistic and multi-trait scoring methods. The results indicated that scores obtained based on the multi-trait rubric were found to be more reliable than those obtained from a holistic rubric. The results also showed that source use was the most reliable feature among all dimensions of the multi-trait rubric. The study results provide sufficient evidence for the use of multi-trait rating scales in the context of scoring RTW tasks. This outcome could encourage more writing instructors to use these scales in their assessment practices given the detailed information they provide about writers' performance on RTW tasks.
Details
- Title: Subtitle
- Integrated writing scores based on holistic and multi-trait scales: A generalizability analysis
- Creators
- Renka Ohta - University of IowaLia M. Plakans - University of IowaAtta Gebril - American University in Cairo
- Resource Type
- Journal article
- Publication Details
- Assessing writing, Vol.38, pp.21-36
- Publisher
- Elsevier
- DOI
- 10.1016/j.asw.2018.08.001
- ISSN
- 1075-2935
- eISSN
- 1873-5916
- Number of pages
- 16
- Grant note
- Graduate Student Research Award from the College of Education at the University of Iowa
- Language
- English
- Date published
- 10/01/2018
- Academic Unit
- Teaching and Learning
- Record Identifier
- 9984371074702771
Metrics
2 Record Views