Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice

Vinay Pursnani; Yusuf Sermet; Kurt Musa; Ibrahim Demir

doi:10.1016/j.caeai.2023.100183

Back

Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice

Journal article

Open access

Peer reviewed

Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice

Vinay Pursnani, Yusuf Sermet, Kurt Musa and Ibrahim Demir

Computers and education. Artificial intelligence, Vol.5, 100183

2023

DOI: 10.1016/j.caeai.2023.100183

Files and links (1)

url

https://doi.org/10.1016/j.caeai.2023.100183View

Published (Version of record) Open Access

Abstract

n recent years, advancements in artificial intelligence (AI) have led to the development of large language models like GPT-4, demonstrating potential applications in various fields, including education. This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. This study further shows a significant improvement in the model's accuracy when answering FE exam questions through noninvasive prompt modifications, substantiating the utility of prompt modification as a viable approach to enhance AI performance in educational contexts. Furthermore, the findings reflect remarkable improvements in mathematical capabilities across successive iterations of ChatGPT models, showcasing their potential in solving complex engineering problems. Our paper also explores future research directions, emphasizing the importance of addressing AI challenges in education, enhancing accessibility and inclusion for diverse student populations, and developing AI-resistant exam questions to maintain examination integrity. By evaluating the performance of ChatGPT in the context of the FE Environmental Exam, this study contributes valuable insights into the potential applications and limitations of large language models in educational settings. As AI continues to evolve, these findings offer a foundation for further research into the responsible and effective integration of AI models across various disciplines, ultimately optimizing the learning experience and improving student outcomes.

Details

Title: Subtitle: Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice
Creators: Vinay Pursnani - University of Iowa
Yusuf Sermet - University of Iowa
Kurt Musa
Ibrahim Demir - University of Iowa
Resource Type: Journal article
Publication Details: Computers and education. Artificial intelligence, Vol.5, 100183
DOI: 10.1016/j.caeai.2023.100183
ISSN: 2666-920X
eISSN: 2666-920X
Grant note: DOI: 10.13039/100000001, name: National Science Foundation; DOI: 10.13039/100000192, name: National Oceanic and Atmospheric Administration
Language: English
Date published: 2023
Academic Unit: Electrical and Computer Engineering; Civil and Environmental Engineering; IIHR--Hydroscience and Engineering; Injury Prevention Research Center
Record Identifier: 9984511759702771

Metrics

6 Record Views