Search papers, labs, and topics across Lattice.
This paper introduces X2Streaming-ASR, a novel approach to streaming automatic speech recognition (ASR) that optimizes both the timing of emissions and the content of those emissions under a hard-commit constraint. By employing a three-stage training procedure that includes establishing recognition ability, warm-starting the commit policy, and refining it with character-level rewards, the system significantly reduces commit latency. The results show that X2Streaming-ASR achieves a mean character-level commit latency of 27-84 ms, outperforming existing streaming ASR systems while also achieving the best streaming character error rate (CER) on multiple datasets.
X2Streaming-ASR slashes commit latency to as low as 27 ms while achieving superior recognition accuracy, setting a new benchmark for streaming ASR systems.
Streaming automatic speech recognition (ASR) for real-time voice agents and full-duplex dialogue must provide accurate partial transcripts with low commit latency. Existing systems commonly use a fixed chunk size, look-ahead, or target delay, or encourage emissions near estimated acoustic boundaries. These approaches do not directly optimize how much additional context to use at each output position under a single-pass, hard-commit constraint. We propose X2Streaming-ASR, which decomposes streaming recognition into when to commit and what to commit. Its three-stage training procedure first establishes streaming recognition ability, then warm-starts the commit policy with automatically probed trajectories, and finally refines the policy using character-level, segment-assigned group-relative rewards for recognition accuracy and latency. Across AISHELL-1/2/3 and WenetSpeech, X2Streaming-ASR achieves a mean character-level commit latency of 27-84 ms relative to forced-aligned character endpoints, compared with 409-585 ms for the evaluated streaming baselines. It achieves the best streaming CER among the evaluated systems on AISHELL-1 and AISHELL-3 with substantially lower latency.
