Unlocking Insights: Investigating Student AI Tutor Interactions in a Large Introductory STEM Course

Jae-Eun Russell; Anna Marie Smith; Salim George; Jonah Pratt; Brian Fodale; Cassandra Monk; Adam Brummett

doi:10.1145/3706468.3706524

Back

Unlocking Insights: Investigating Student AI Tutor Interactions in a Large Introductory STEM Course

Conference proceeding

Open access

Unlocking Insights: Investigating Student AI Tutor Interactions in a Large Introductory STEM Course

Jae-Eun Russell, Anna Marie Smith, Salim George, Jonah Pratt, Brian Fodale, Cassandra Monk and Adam Brummett

Proceedings of the 15th International Learning Analytics and Knowledge Conference, pp.451-461

ACM Other Conferences

LAK '25: The 15th International Learning Analytics and Knowledge Conference

03/03/2025

DOI: 10.1145/3706468.3706524

Appears in UI Libraries Support Open Access

Files and links (1)

url

https://doi.org/10.1145/3706468.3706524View

Published (Version of record) Open Access

Abstract

This study explored the use of an AI tutor and its relationship to performance outcomes in a large introductory undergraduate STEM course, where the AI tutor was integrated into the online homework system. The course included 13 weekly homework assignments, comprising 221 questions that contributed 19.5% to the final grade. Results showed that students predominantly completed homework problems without AI tutor assistance, using it selectively to address specific challenges. Patterns of AI interaction varied at both the problem and student levels, with demographic factors having little to no relationship to AI usage. Notably, the frequency of AI use was not linked to exam performance. A multi-level cluster analysis identified distinct patterns in students’ use of the AI tutor during problem-solving. These patterns of use had more significant associations with performance than frequency of use alone. This paper explores these interaction patterns in depth and discusses the study’s limitations and implications.

Applied computing -- Computer-assisted instruction

Applied computing -- E-learning

Applied computing -- Interactive learning environments

Human-centered computing -- Empirical studies in HCI

UIOWA OA Agreement

Details

Title: Subtitle: Unlocking Insights: Investigating Student AI Tutor Interactions in a Large Introductory STEM Course
Creators: Jae-Eun Russell - University of Iowa, ITS Teaching, Learning & Technology
Anna Marie Smith - University of Iowa, ITS Teaching, Learning & Technology
Salim George - University of Iowa
Jonah Pratt - University of Iowa
Brian Fodale - University of Iowa
Cassandra Monk - University of Iowa
Adam Brummett - University of Iowa
Resource Type: Conference proceeding
Publication Details: Proceedings of the 15th International Learning Analytics and Knowledge Conference, pp.451-461
Conference: LAK '25: The 15th International Learning Analytics and Knowledge Conference
Series: ACM Other Conferences
DOI: 10.1145/3706468.3706524
Publisher: Association for Computing Machinery
Number of pages: 11
Language: English
Date published: 03/03/2025
Academic Unit: ITS Teaching, Learning & Technology; Chemistry; University College Courses
Record Identifier: 9984821675002771

Metrics

21 Record Views

1 Times Cited - Web of Science