Journal article
Building an ICCN Multimodal Classifier of Aggressive Political Debate Style: Towards a Computational Understanding of Candidate Performance Over Time
Communication methods and measures, Vol.ahead-of-print(ahead-of-print), pp.1-18
06/23/2023
DOI: 10.1080/19312458.2023.2227093
Abstract
Understanding the implications of aggressive political debate style amid corrosive modes of campaign politics requires fine-grained analyses of political performance, attending to multiple communication modalities. Politicians' facial expressions, emotional tone, and speech content can all independently convey aggression and dominance, and often work in combination for purposes of emphasis. Yet micro-coding individual visual, tonal, and verbal features across more than a handful of debate segments becomes extremely labor intensive, hampering research, especially historical, longitudinal, and cross-cultural work. To address this limitation, we develop a novel multimodal classifier using an Interaction Canonical Correlation Network (ICCN) that incorporates video and audio features with speech coding of candidate debate performance, trained on a 20% sample of 10-second segments from each of the first televised U.S presidential debates between 1980 and 2020. In the analysis, we demonstrate this classifier can accurately detect aggression by political candidates in U.S. debates. We sharpen its performance by distinguishing between debate eras characterized by lower and higher levels of aggression and validate the approach by comparing the performance of unimodal with multimodal classification. This classifier opens new avenues for computational social science research, including explaining candidate behavior within debates at a larger scale and across different eras.
Details
- Title: Subtitle
- Building an ICCN Multimodal Classifier of Aggressive Political Debate Style: Towards a Computational Understanding of Candidate Performance Over Time
- Creators
- Dhavan V. Shah - University of Wisconsin–MadisonZhongkai Sun - Amazon Alexa AIErik P. Bucy - Texas Tech UniversitySang Jung Kim - University of IowaYibing Sun - University of Wisconsin–MadisonMengyu Li - University of Wisconsin–MadisonWilliam Sethares - University of Wisconsin–Madison
- Resource Type
- Journal article
- Publication Details
- Communication methods and measures, Vol.ahead-of-print(ahead-of-print), pp.1-18
- Publisher
- Routledge
- DOI
- 10.1080/19312458.2023.2227093
- ISSN
- 1931-2458
- eISSN
- 1931-2466
- Grant note
- DOI: 10.13039/501100003725, name: National Research Foundation of Korea; DOI: 10.13039/100007131, name: Texas Tech University
- Language
- English
- Electronic publication date
- 06/23/2023
- Academic Unit
- School of Journalism and Mass Communication; Center for Social Science Innovation
- Record Identifier
- 9984459635102771
Metrics
4 Record Views