Journal article
Facial Expressions as an Index of Listening Difficulty and Emotional Response
Seminars in hearing, Vol.44(2), pp.166-187
05/2023
DOI: 10.1055/s-0043-1766104
PMCID: PMC10147507
PMID: 37122878
Abstract
Abstract
Knowledge about listening difficulty experienced during a task can be used to better understand speech perception processes, to guide amplification outcomes, and can be used by individuals to decide whether to participate in communication. Another factor affecting these decisions is individuals' emotional response which has not been measured objectively previously. In this study, we describe a novel method of measuring listening difficulty and affect of individuals in adverse listening situations using automatic facial expression algorithm. The purpose of our study was to determine if facial expressions of confusion and frustration are sensitive to changes in listening difficulty. We recorded speech recognition scores, facial expressions, subjective listening effort scores, and subjective emotional responses in 33 young participants with normal hearing. We used the signal-to-noise ratios of −1, +2, and +5 dB SNR and quiet conditions to vary the difficulty level. We found that facial expression of confusion and frustration increased with increase in difficulty level, but not with change in each level. We also found a relationship between facial expressions and both subjective emotion ratings and subjective listening effort. Emotional responses in the form of facial expressions show promise as a measure of affect and listening difficulty. Further research is needed to determine the specific contribution of affect to communication in challenging listening environments.
Details
- Title: Subtitle
- Facial Expressions as an Index of Listening Difficulty and Emotional Response
- Creators
- Soumya Venkitakrishnan - Department of Communication Sciences and Disorders, University of IowaYu-Hsiang Wu - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Seminars in hearing, Vol.44(2), pp.166-187
- Publisher
- Thieme Medical Publishers, Inc
- DOI
- 10.1055/s-0043-1766104
- PMID
- 37122878
- PMCID
- PMC10147507
- ISSN
- 0734-0451
- eISSN
- 1098-8955
- Language
- English
- Electronic publication date
- 04/04/2023
- Date published
- 05/2023
- Academic Unit
- Communication Sciences and Disorders
- Record Identifier
- 9984386255002771
Metrics
14 Record Views