Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper tackles device-addressed speech detection for on-device voice AI, framing it as a Sequential Device-Addressed Routing (SDAR) problem to leverage interaction history. They introduce the Selective Attention System (SAS), an on-device implementation that uses attention mechanisms to selectively route audio based on interaction history. Results on a 60-hour multi-speaker dataset show that SAS achieves F1 scores of 0.86 (audio-only) and 0.95 (audio+video), with interaction history proving crucial for performance.
Ignoring interaction history in device-addressed speech detection can slash F1 scores by nearly 40%, highlighting the importance of sequential routing.
We study device-addressed speech detection under pre-ASR edge deployment constraints, where systems must decide whether to forward audio before transcription under strict latency and compute limits. We show that, in multi-speaker environments with temporally ambiguous utterances, this task is more effectively modelled as a sequential routing problem over interaction history than as an utterance-local classification task. We formalize this as Sequential Device-Addressed Routing (SDAR) and present the Selective Attention System (SAS), an on-device implementation that instantiates this formulation. On a held-out 60-hour multi-speaker English test set, the primary audio-only configuration achieves F1=0.86 (precision=0.89, recall=0.83); with an optional camera, audio+video fusion raises F1 to 0.95 (precision=0.97, recall=0.93). Removing causal interaction history (Stage~3) reduced F1 from 0.95 to 0.57+/-0.03 in the audio+video configuration under our evaluation protocol. Among the tested components, this was the largest observed ablation effect, indicating that short-horizon interaction history carries substantial decision-relevant information in the evaluated setting. SAS runs fully on-device on ARM Cortex-A class hardware (<150 ms latency,<20 MB footprint). All results are from internal evaluation on a proprietary dataset evaluated primarily in English; a 5-hour evaluation subset may be shared for independent verification (Section 8.8).