Journal article
Cognitive validity in listen-to-write summary tasks: A mixed-methods analysis of notetaking data
Studies in Language Assessment, Vol.15(1), pp.105-141
09/2026
DOI: 10.58379/SPQD9576
Abstract
This study examines the cognitive validity of a listen-to-summarize task on a university placement test. Cognitive validity is underinvestigated in listening assessment, especially for higher-order listening processes (Holzknecht et al., 2017; Rukthong, 2021). However, student notes from listening tests can record evidence of these processes (Yeager et al., 2024; Cushing, 1991). This study employed an exploratory sequential mixed-methods design to investigate the higher-order listening processes elicited by an integrated listen-to-summarize task. First, notes from two administrations of the same test form – one with notetaking scaffolding (n = 25) and one without (n = 25) – were analyzed qualitatively. Second, qualitative codes informed the development of a rubric to quantify higher-order listening processes across administrations, enabling comparison with summary and multiple-choice scores on another test form (n = 118). Qualitative analysis revealed substantial evidence of higher-order discourse construction processes in student notes, including Selecting, Integrating, Monitoring, and Structure-Building (Field, 2013), especially when scaffolding was provided. Linear regression models indicated that Selecting processes significantly predicted test scores for both the integrated task and the multiple-choice task, while Structure-Building processes were significant only for the integrated task. Results support the cognitive validity of the listen-to-summarize task and inform improved test design and score interpretation.
Details
- Title: Subtitle
- Cognitive validity in listen-to-write summary tasks: A mixed-methods analysis of notetaking data
- Creators
- Rebecca Yeager - University of Illinois Urbana-ChampaignGoMee Park - The University of Texas Rio Grande ValleyJunhee Park - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Studies in Language Assessment, Vol.15(1), pp.105-141
- DOI
- 10.58379/SPQD9576
- ISSN
- 2653-5335
- eISSN
- 2653-5335
- Number of pages
- 37
- Language
- English
- Date published
- 09/2026
- Academic Unit
- Languages, Linguistics, Literatures, and Cultures
- Record Identifier
- 9985230929202771
Metrics
1 Record Views