Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Gavin Fulmer; Hye-Eun Chu; David Treagust; Knut Neumann

doi:10.1186/s41029-015-0005-x

Back

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Journal article

Open access

Peer reviewed

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Gavin Fulmer, Hye-Eun Chu, David Treagust and Knut Neumann

Asia-Pacific Science Education, Vol.1(1), pp.1-16

12/2015

DOI: 10.1186/s41029-015-0005-x

Files and links (1)

url

https://doi.org/10.1186/s41029-015-0005-xView

Published (Version of record) Open Access

Abstract

Two-tier multiple-choice (TTMC) items are used to assess students’ knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2. But are the knowledge and reasoning involved in these tiers really distinguishable? Are the tiers equally challenging for students? The answers to these questions influence how we use and interpret TTMC instruments. We apply the Rasch measurement model on TTMC items to see if the items are distinguishable according to different traits (represented by the tier), or according to different content sub-topics within the instrument, or to both content and tier. Two TTMC data sets are analyzed: data from Singapore and Korea on the Light Propagation Diagnostic Instrument (LPDI), data from the United States on the Classroom Test of Scientific Reasoning (CTSR). Findings for LPDI show that tier-2 reasoning items are more difficult than tier-1 knowledge items, across content sub-topics. Findings for CTSR do not show a consistent pattern by tier or by content sub-topic. We conclude that TTMC items cannot be assumed to have a consistent pattern of difficulty by tier—and that assessment developers and users need to consider how the tiers operate when administering TTMC items and interpreting results. Researchers must check the tiers’ difficulties empirically during validation and use. Though findings from data in Asian contexts were more consistent, further study is needed to rule out differences between the LPDI and CTSR instruments.

Education

Optics

Science Education

Learning & Instruction

Teaching and Teacher Education

Two-tier items

Rasch measurement models

Scientific reasoning

Details

Title: Subtitle: Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model
Creators: Gavin Fulmer - National Institute of Education, Nanyang Technological University ᅟ Singapore
Hye-Eun Chu - School of Education Macquarie University Sydney Australia
David Treagust - Science & Mathematics Education Centre Curtin University Perth Australia
Knut Neumann - Leibniz Institute for Science and Mathematics Education (IPN), University of Kiel Kiel Germany
Resource Type: Journal article
Publication Details: Asia-Pacific Science Education, Vol.1(1), pp.1-16
DOI: 10.1186/s41029-015-0005-x
ISSN: 2364-1177
eISSN: 2364-1177
Publisher: Springer International Publishing; Cham
Language: English
Date published: 12/2015
Academic Unit: Teaching and Learning
Record Identifier: 9983993498902771

Metrics

27 Record Views