Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR

Zelin Wu; Gan Song; Christopher Li; Pat Rondon; Zhong Meng; Xavier Velez; Weiran Wang; Diamantino Caseiro; Golan Pundak; Tsendsuren Munkhdalai; Angad Chandorkar; Rohit Prabhavalkar

doi:10.48550/arxiv.2404.10180

Back

Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR

Preprint

Open access

Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR

Zelin Wu, Gan Song, Christopher Li, Pat Rondon, Zhong Meng, Xavier Velez, Weiran Wang, Diamantino Caseiro, Golan Pundak, Tsendsuren Munkhdalai, …

arXiv.org

04/23/2024

DOI: 10.48550/arxiv.2404.10180

Files and links (1)

url

https://doi.org/10.48550/arXiv.2404.10180View

Preprint (Author's original)This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

Contextual biasing enables speech recognizers to transcribe important phrases in the speaker's context, such as contact names, even if they are rare in, or absent from, the training data. Attention-based biasing is a leading approach which allows for full end-to-end cotraining of the recognizer and biasing system and requires no separate inference-time components. Such biasers typically consist of a context encoder; followed by a context filter which narrows down the context to apply, improving per-step inference time; and, finally, context application via cross attention. Though much work has gone into optimizing per-frame performance, the context encoder is at least as important: recognition cannot begin before context encoding ends. Here, we show the lightweight phrase selection pass can be moved before context encoding, resulting in a speedup of up to 16.1 times and enabling biasing to scale to 20K phrases with a maximum pre-decoding delay under 33ms. With the addition of phrase- and wordpiece-level cross-entropy losses, our technique also achieves up to a 37.5% relative WER reduction over the baseline without the losses and lightweight phrase selection pass.

Computer Science - Artificial Intelligence

Computer Science - Computation and Language

Computer Science - Learning

Computer Science - Neural and Evolutionary Computing

Details

Title: Subtitle: Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR
Creators: Zelin Wu
Gan Song
Christopher Li
Pat Rondon
Zhong Meng
Xavier Velez
Weiran Wang
Diamantino Caseiro
Golan Pundak
Tsendsuren Munkhdalai
Angad Chandorkar
Rohit Prabhavalkar
Resource Type: Preprint
Publication Details: arXiv.org
DOI: 10.48550/arxiv.2404.10180
eISSN: 2331-8422
Number of pages: 9 pages
Comment: accepted by 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics - Industry Track
Language: English
Date posted: 04/23/2024
Academic Unit: Computer Science
Record Identifier: 9984696797502771

Metrics

9 Record Views