Dissertation
Empirical investigation of biometric and behavioral indicators of human performance in bridge inspection with drone and damage detection with artificial intelligence assistance
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Spring 2024
DOI: 10.25820/etd.007407
Abstract
The integration of Artificial Intelligence (AI) and drone technology into bridge inspections represents a significant advancement in infrastructure assessment methodologies, offering enhanced efficiency and safety. However, this integration introduces changes in the cognitive ergonomics of the inspection task, necessitating a deeper understanding of human-drone-AI interaction dynamics. This thesis explores the cognitive and behavioral aspects of human performance in AI-assisted drone-enabled bridge inspections through a series of experiments.
The first investigation focuses on underlying affective behaviors related to drone pilot skill acquisition, employing Electroencephalography (EEG) and Eye tracking instrumentation to measure human affect during simulated drone piloting tasks. Results highlight the impact of task difficulty on performance/learning processes and underscore the importance of tailoring drone specifications and training requirements to mission-specific contexts. Furthermore, psychological and behavioral measures are identified as theoretical foundations for modeling complex tasks.
Building upon these insights, the subsequent experiment examines human performance in drone-enabled bridge inspections under two conditions: with AI assistance and without. Biometric and behavioral data collected during these simulations reveal that different cognitive states can influence inspectors' performance respectively in either condition. This highlights the importance of designing inspection protocols, drones and AI systems based on a comprehensive understanding of the cognitive processes required in each condition to prevent cognitive overload and minimize errors. This study also identifies effective visual scanning and gaze patterns in each condition that inspectors can implement to enhance their inspection performance.
Finally, this thesis proposes a cognitive monitoring framework designed to detect when bridge inspectors might miss crucial damage, using EEG readings, eye movement tracking, and controller input data. Utilizing spectrograms obtained from EEG signals, eye movement data, and inputs from drone controllers, Convolutional Neural Networks (CNNs) alongside various data fusion methods were employed to boost the predictive accuracy in identifying inspectors’ failures to detect cracks.
Overall, this research contributes to advancing human-automation cooperation for bridge inspections, offering insights into optimizing inspector training, workflow design, and cognitive monitoring strategies.
Details
- Title: Subtitle
- Empirical investigation of biometric and behavioral indicators of human performance in bridge inspection with drone and damage detection with artificial intelligence assistance
- Creators
- Fatemeh Dalilian
- Contributors
- David Nembhard (Advisor)Geb Thomas (Committee Member)Daniel McGehee (Committee Member)Andrea Luangrath (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Industrial Engineering
- Date degree season
- Spring 2024
- Publisher
- University of Iowa
- DOI
- 10.25820/etd.007407
- Number of pages
- xiii, 182 pages
- Copyright
- Copyright 2024 Fatemeh Dalilian
- Grant note
- I would also like to acknowledge the support received from the National Science Foundation (NSF), which has been instrumental in facilitating much of the research presented in this thesis (iii)
- Language
- English
- Date submitted
- 04/11/2024
- Description illustrations
- illustrations, tables, graphs
- Description bibliographic
- Includes bibliographical references (pages 140-182).
- Public Abstract (ETD)
- A considerable proportion of bridges across the United States require maintenance or repairs to ensure their continued safety and functionality. As such, the development and implementation of effective bridge inspection and maintenance strategies are essential for sustaining the efficiency of our transportation infrastructure. Traditional methods of inspecting bridges can be slow, costly, and prone to oversight. To address these issues, advancements in technology have led to the development of AI-powered drones for bridge inspections. Human inspectors utilize these innovative drones to navigate different sections of bridges capturing intricate images and data using advanced cameras, sensors, and damage detection AI algorithms. This allows them to identify potential issues such as cracks or corrosion efficiently and comprehensively. This study delves into how humans interact with drones and damage detection AIs for bridge inspections. Through carefully designed experiments, the research uncovers how cognitive and behavioral factors relate to inspectors' effectiveness in operating drones and detecting cracks during bridge inspections. Subsequently, a monitoring system is designed that can detect when bridge inspectors and trainees are prone to mistakes during inspections. This proactive approach ensures that inspectors remain focused and safe while operating this new technology. This study's findings serve as a roadmap for the advancement of bridge inspection technologies and the development of future inspector training programs. Moreover, this research sets the foundation for the establishment of cognitive monitoring systems in environments where humans collaborate with robots.
- Academic Unit
- Industrial and Systems Engineering
- Record Identifier
- 9984647455502771
Metrics
4 File views/ downloads
16 Record Views