Logo image
Back
Massive End-to-end Speech Recognition Models with Time Reduction
Conference proceeding   Open access

Massive End-to-end Speech Recognition Models with Time Reduction

Weiran Wang, Rohit Prabhavalkar, Haozhe Shan, Zhong Meng, Dongseong Hwang, Qiujia Li, Khe Chai Sim, Bo Li, James Qin, Xingyu Cai, …
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp.6206-6217
01/01/2024
DOI: 10.18653/v1/2024.naacl-long.344
url
https://doi.org/10.18653/v1/2024.naacl-long.344View
Published (Version of record) Open Access

Abstract

We investigate massive end-to-end automatic speech recognition (ASR) models with efficiency improvements achieved by time reduction. The encoders of our models use the neural architecture of Google's universal speech model (USM), with additional funnel pooling layers to significantly reduce the frame rate and speed up training and inference. We also explore a few practical methods to mitigate potential accuracy loss due to time reduction, while enjoying most efficiency gain. Our methods are demonstrated to work with both Connectionist Temporal Classification (CTC) and RNN-Transducer (RNN-T), with up to 2B model parameters, and over two domains. For a large-scale voice search recognition task, we perform extensive studies on vocabulary size, time reduction strategy, and its generalization performance on long-form test sets, and show that a 900M RNN-T is very tolerant to severe time reduction, with as low encoder output frame rate as 640ms. We also provide ablation studies on the Librispeech benchmark for important training hyperparameters and architecture designs, in training 600M RNN-T models at the frame rate of 160ms.
 Computer Science Social Sciences Technology Computer Science, Artificial Intelligence Computer Science, Interdisciplinary Applications Linguistics Science & Technology

Details

Metrics

2 Record Views
Logo image