Conference proceeding
SIGNER-INDEPENDENT FINGERSPELLING RECOGNITION WITH DEEP NEURAL NETWORK ADAPTATION
2016 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING PROCEEDINGS, Vol.2016-, pp.6160-6164
International Conference on Acoustics Speech and Signal Processing ICASSP
03/01/2016
DOI: 10.1109/ICASSP.2016.7472861
Abstract
We study the problem of recognition of fingerspelled letter sequences in American Sign Language in a signer-independent setting. Fingerspelled sequences are both challenging and important to recognize, as they are used for many content words such as proper nouns and technical terms. Previous work has shown that it is possible to achieve almost 90% accuracies on fingerspelling recognition in a signer-dependent setting. However, the more realistic signer-independent setting presents challenges due to significant variations among signers, coupled with the dearth of available training data. We investigate this problem with approaches inspired by automatic speech recognition. We start with the best-performing approaches from prior work, based on tandem models and segmental conditional random fields (SCRFs), with features based on deep neural network (DNN) classifiers of letters and phonological features. Using DNN adaptation, we find that it is possible to bridge a large part of the gap between signer-dependent and signer-independent performance. Using only about 115 transcribed words for adaptation from the target signer, we obtain letter accuracies of up to 82.7% with framelevel adaptation labels and 69.7% with only word labels.
Details
- Title: Subtitle
- SIGNER-INDEPENDENT FINGERSPELLING RECOGNITION WITH DEEP NEURAL NETWORK ADAPTATION
- Creators
- Taehwan Kim - Toyota Technol Inst Chicago, Chicago, IL 60637 USAWeiran Wang - Toyota Technological Institute at ChicagoHao Tang - Toyota Technological Institute at ChicagoKaren Livescu - Toyota Technological Institute at Chicago
- Resource Type
- Conference proceeding
- Publication Details
- 2016 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING PROCEEDINGS, Vol.2016-, pp.6160-6164
- Publisher
- IEEE
- Series
- International Conference on Acoustics Speech and Signal Processing ICASSP
- DOI
- 10.1109/ICASSP.2016.7472861
- ISSN
- 1520-6149
- eISSN
- 2379-190X
- Number of pages
- 5
- Language
- English
- Date published
- 03/01/2016
- Academic Unit
- Computer Science
- Record Identifier
- 9984696579102771
Metrics
6 Record Views