| Paper ID | SPE-10.4 |
| Paper Title |
DEVELOPING REAL-TIME STREAMING TRANSFORMER TRANSDUCER FOR SPEECH RECOGNITION ON LARGE-SCALE DATASET |
| Authors |
Xie Chen, Yu Wu, Zhenghao Wang, Shujie Liu, Jinyu Li, Microsoft, United States |
| Session | SPE-10: Speech Recognition 4: Transformer Models 2 |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 16:30 - 17:15 |
| Presentation Time: | Tuesday, 08 June, 16:30 - 17:15 |
| Presentation |
Poster
|
| Topic |
Speech Processing: [SPE-LVCR] Large Vocabulary Continuous Recognition/Search |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Recently, Transformer based end-to-end models have achieved great success in many areas including speech recognition. However, compared to LSTM models, the heavy computational cost of the Transformer during inference is a potential issue to prevent their applications. In this work, we explored the potential of Transformer Transducer (T-T) models for the fist pass decoding with low latency and fast speed on a large-scale dataset. We combine the idea of Transformer-XL and chunk-wise streaming processing to design a streamable Transformer Transducer model. We demonstrate that T-T outperforms the hybrid model, RNN Transducer (RNN-T), and streaming Transformer attention-based encoder-decoder model in the streaming scenario. Furthermore, the runtime cost and latency can be optimized with a relatively small look-ahead. |
