Speech & Audio - Weekly Roundup

Beyond transcribing words, a novel convolutional-Transformer model now timestamps non-verbal vocalizations and background sounds, enriching ASR with a richer auditory context.

Self-supervised speech embeddings unlock substantial gains in cross-speaker dysarthria detection during awake craniotomy, outperforming handcrafted features by up to 26.1% AUC.

Frontier MLLMs exhibit strong local perception but falter on long-horizon audio-visual reasoning, with a single benchmark exposing specific failure modes like identity drift and cross-modal misalignment.

Complex-valued fusion of audio-visual embeddings unlocks significant gains in open-vocabulary event localization, surpassing prior methods by up to 6.6% Event-F1.

Decouple complex audio scenes into individual, controllable sound sources directly from a single generative model.

Beyond transcribing words, a novel convolutional-Transformer model now timestamps non-verbal vocalizations and background sounds, enriching ASR with a richer auditory context.

Self-supervised speech embeddings unlock substantial gains in cross-speaker dysarthria detection during awake craniotomy, outperforming handcrafted features by up to 26.1% AUC.

Frontier MLLMs exhibit strong local perception but falter on long-horizon audio-visual reasoning, with a single benchmark exposing specific failure modes like identity drift and cross-modal misalignment.

Complex-valued fusion of audio-visual embeddings unlocks significant gains in open-vocabulary event localization, surpassing prior methods by up to 6.6% Event-F1.

Decouple complex audio scenes into individual, controllable sound sources directly from a single generative model.

Audio-language models excel at identifying sounds but fail to grasp the crucial context of how speakers react to them.

Fine-grained emotion control in TTS is now achievable at inference time with minimal speaker or content degradation, bypassing the need for expensive model retraining.

Black-box text-to-music generators can be steered to precisely match complex user intent by an agent that iteratively refines prompts based on rubric-based audio verification.

Latency in voice assistants plummets by 20% by speculatively executing tool calls *during* speech recognition, not after.

Interactive AI worlds can now speak and react in real-time with spatially accurate sound, bridging the sensory gap in generative environments.

A single-voice TTS model with 10x fewer parameters and 15x less compute than its teacher achieves near-identical audio quality, runnable on a single laptop.

Wrist accelerometers can now yield population-scale sleep heart rate insights, with a novel transformer model outperforming existing methods in estimation availability.

A novel cross-domain recommendation framework achieves state-of-the-art book-to-music mood matching by fusing sentiment embeddings with LLM-driven emotional quadrant classification.

Simple vMF likelihood modeling unlocks robust label-free speaker embedding enhancement, outperforming complex diffusion baselines on challenging acoustic mismatches.