Machine Learning Mixed Methods Text Analysis: An Illustration From Automated Scoring Models of Student Writing in Biology Education

Kamali N. Sripathi; Rosa A. Moscarella; Matthew Steele; Rachel Yoho; Hyesun You; Luanna B. Prevost; Mark Urban-Lurain; John Merrill; Kevin C. Haudek

doi:10.1177/15586898231153946

Back

Machine Learning Mixed Methods Text Analysis: An Illustration From Automated Scoring Models of Student Writing in Biology Education

Journal article

Open access

Peer reviewed

Machine Learning Mixed Methods Text Analysis: An Illustration From Automated Scoring Models of Student Writing in Biology Education

Kamali N. Sripathi, Rosa A. Moscarella, Matthew Steele, Rachel Yoho, Hyesun You, Luanna B. Prevost, Mark Urban-Lurain, John Merrill and Kevin C. Haudek

Journal of mixed methods research, Vol.18(1), pp.48-70

01/2024

DOI: 10.1177/15586898231153946

Files and links (1)

url

https://doi.org/10.1177/15586898231153946View

Published (Version of record) Open Access

Abstract

Assessing student knowledge based on their writing using traditional qualitative methods is time-consuming. To improve speed and consistency of text analysis, we present our mixed methods development of a machine learning predictive model to analyze student writing. Our approach involves two stages: first an exploratory sequential design, and second an iterative complex design. We first trained our predictive model using qualitative coding of categories (ideas) in student writing. We next revised our model based on feedback from instructor-users. The model itself highlighted categories in need of revision. The contribution to mixed methods research lies in our innovative use of the machine learning tool as a rapid, consistent additional coder, and a resource that can predict codes for new student writing.

Machine Learning

biology assessment

predictive models

constructed response assessments

biology education research

Details

Title: Subtitle: Machine Learning Mixed Methods Text Analysis: An Illustration From Automated Scoring Models of Student Writing in Biology Education
Creators: Kamali N. Sripathi - University of California, Davis
Rosa A. Moscarella - University of Massachusetts Amherst
Matthew Steele - Michigan State University
Rachel Yoho - George Mason University
Hyesun You - University of Iowa
Luanna B. Prevost - University of South Florida
Mark Urban-Lurain - Michigan State University
John Merrill - Michigan State University
Kevin C. Haudek - Michigan State University
Resource Type: Journal article
Publication Details: Journal of mixed methods research, Vol.18(1), pp.48-70
DOI: 10.1177/15586898231153946
ISSN: 1558-6898
eISSN: 1558-6901
Publisher: SAGE Publications
Grant note: 1323162 / National Science Foundation (https://doi.org/10.13039/100000001) 1347740 / National Science Foundation (https://doi.org/10.13039/100000001)
Language: English
Electronic publication date: 02/03/2023
Date published: 01/2024
Academic Unit: Teaching and Learning
Record Identifier: 9984371109602771

Metrics

32 Record Views

17 Times Cited - Web of Science

See more details