Journal article
Modeling the Development of Audiovisual Cue Integration in Speech Perception
Brain sciences, Vol.7(3), p.32
03/21/2017
DOI: 10.3390/brainsci7030032
PMCID: PMC5366831
PMID: 28335558
Abstract
Adult speech perception is generally enhanced when information is provided from multiple modalities. In contrast, infants do not appear to benefit from combining auditory and visual speech information early in development. This is true despite the fact that both modalities are important to speech comprehension even at early stages of language acquisition. How then do listeners learn how to process auditory and visual information as part of a unified signal? In the auditory domain, statistical learning processes provide an excellent mechanism for acquiring phonological categories. Is this also true for the more complex problem of acquiring audiovisual correspondences, which require the learner to integrate information from multiple modalities? In this paper, we present simulations using Gaussian mixture models (GMMs) that learn cue weights and combine cues on the basis of their distributional statistics. First, we simulate the developmental process of acquiring phonological categories from auditory and visual cues, asking whether simple statistical learning approaches are sufficient for learning multi-modal representations. Second, we use this time course information to explain audiovisual speech perception in adult perceivers, including cases where auditory and visual input are mismatched. Overall, we find that domain-general statistical learning techniques allow us to model the developmental trajectory of audiovisual cue integration in speech, and in turn, allow us to better understand the mechanisms that give rise to unified percepts based on multiple cues.
Details
- Title: Subtitle
- Modeling the Development of Audiovisual Cue Integration in Speech Perception
- Creators
- Laura M. Getz - Villanova UniversityElke R. Nordeen - Villanova UniversitySarah C. Vrabic - Villanova UniversityJoseph C. Toscano - Villanova University
- Resource Type
- Journal article
- Publication Details
- Brain sciences, Vol.7(3), p.32
- Publisher
- Mdpi
- DOI
- 10.3390/brainsci7030032
- PMID
- 28335558
- PMCID
- PMC5366831
- ISSN
- 2076-3425
- eISSN
- 2076-3425
- Number of pages
- 23
- Grant note
- Mendel Science Experience Post-doctoral Fellowship from the Villanova University College of Liberal Arts and Sciences
- Language
- English
- Date published
- 03/21/2017
- Academic Unit
- Psychological and Brain Sciences
- Record Identifier
- 9984627200802771
Metrics
5 Record Views