Search papers, labs, and topics across Lattice.
This paper introduces Agentic Speech Recognition (AgenticSR), which addresses the limitations of traditional automatic speech recognition (ASR) by refining transcriptions in real-time to eliminate disfluencies and self-corrections while preserving the speaker's intent. The proposed ASR-Refiner architecture allows for continual updates to the text output as new audio is processed, significantly enhancing usability in real-world scenarios. Evaluation on the AASR-Bench benchmark demonstrates that AgenticASR outperforms existing systems, achieving the highest scores and aligning well with expert assessments.
AgenticASR revolutionizes speech recognition by enabling real-time, intent-preserving transcription that adapts as speech evolves, outperforming traditional methods.
Automatic speech recognition (ASR) has achieved substantial gains in transcription accuracy, yet verbatim transcription does not necessarily produce readily usable text. It retains fillers, repetitions, false starts, and self-corrections that increase reading effort, obscure the speaker's final intent, and propagate unresolved or abandoned content to downstream tasks. Existing spoken-to-written methods process completed audio or transcripts but cannot revise emitted text when later speech changes how preceding content should be interpreted. We therefore formulate Agentic Speech Recognition (AgenticSR), an audio-to-clean-text task that removes disfluencies, resolves self-corrections, and normalizes written form while preserving the speaker's final intent. AgenticASR implements this task through an ASR--Refiner architecture that repeatedly transforms a bounded active context and replaces its corresponding output span as audio arrives. This enables continual emission and revision over streams of arbitrary duration. We also introduce AASR-Bench, a bilingual benchmark with fine-grained atomic rubrics. Across multiple ASR front ends, AgenticASR attains the highest AASR-Bench scores among evaluated systems. A human--AI agreement study shows that rubric-based judgments align with independent expert assessments. Ablations characterize Refiner capacity, context length, and the quality--latency trade-off between online and offline inference. Together, these results establish AgenticASR as a practical framework for intent-preserving clean transcription during ongoing speech. Code, AASR-Bench, and a demo will be released at https://github.com/AnXMuy/AgenticASR.
