Journal article
Question preview in English for academic purposes listening assessment: the effect of stem preview on difficulty, item type, and discrimination
International journal of listening, Vol.36(3), pp.299-324
09/2022
DOI: 10.1080/10904018.2022.2029705
Abstract
This study investigates the effects of adding stem preview to an English for Academic Purposes (EAP) multiple-choice listening assessment. In stem preview, listeners may view the item stems, but not response options, before listening. Previous research indicates that adding preview to an exam typically decreases difficulty, but raises concerns about score interpretation. Concerningly, no previous studies have explored the impact of preview on item discrimination, a key assumption of a validity argument. Our study utilized a Latin square design controlling for group, lecture, and preview condition to explore the impact of stem preview on difficulty, item type, and discrimination. Analysis indicated no significant effects of preview condition on difficulty or item type overall at our chosen alpha level. However, comparisons of total scores revealed a bimodal distribution in the no-preview condition, but not in the preview condition, indicating lower-scoring students received a boost from stem preview. Additionally, preview significantly increased facility on one of the trivial items. Results for discrimination were more complicated. On two of the five discrimination indices in the study, preview significantly decreased item discrimination, but for the other three indices, effects were not significant. Implications for assessment developers and researchers are discussed.
Details
- Title: Subtitle
- Question preview in English for academic purposes listening assessment: the effect of stem preview on difficulty, item type, and discrimination
- Creators
- Rebecca Yeager - University of Iowa, English as a Second LanguageZachary Meyer - University of Iowa, English as a Second Language
- Resource Type
- Journal article
- Publication Details
- International journal of listening, Vol.36(3), pp.299-324
- DOI
- 10.1080/10904018.2022.2029705
- ISSN
- 1090-4018
- eISSN
- 1932-586X
- Publisher
- Routledge
- Language
- English
- Electronic publication date
- 02/03/2022
- Date published
- 09/2022
- Academic Unit
- English as a Second Language
- Record Identifier
- 9984296647902771
Metrics
31 Record Views