Speech & Audio - Weekly Roundup

A single-voice TTS model with 10x fewer parameters and 15x less compute than its teacher achieves near-identical audio quality, runnable on a single laptop.

Wrist accelerometers can now yield population-scale sleep heart rate insights, with a novel transformer model outperforming existing methods in estimation availability.

A novel cross-domain recommendation framework achieves state-of-the-art book-to-music mood matching by fusing sentiment embeddings with LLM-driven emotional quadrant classification.

Simple vMF likelihood modeling unlocks robust label-free speaker embedding enhancement, outperforming complex diffusion baselines on challenging acoustic mismatches.

Iterative refinement of a class token with shared blocks slashes transformer parameters and computation while boosting audio recognition accuracy.

A single-voice TTS model with 10x fewer parameters and 15x less compute than its teacher achieves near-identical audio quality, runnable on a single laptop.

Wrist accelerometers can now yield population-scale sleep heart rate insights, with a novel transformer model outperforming existing methods in estimation availability.

A novel cross-domain recommendation framework achieves state-of-the-art book-to-music mood matching by fusing sentiment embeddings with LLM-driven emotional quadrant classification.

Simple vMF likelihood modeling unlocks robust label-free speaker embedding enhancement, outperforming complex diffusion baselines on challenging acoustic mismatches.

Iterative refinement of a class token with shared blocks slashes transformer parameters and computation while boosting audio recognition accuracy.

Acoustic noise, even when not diagnostically informative, can systematically manipulate Alzheimer's predictions in SSL models, revealing a critical vulnerability in current speech-based assessments.

Zero-shot voice conversion now achieves $9\times$ faster inference with improved perceptual quality by jointly optimizing flow and content encoding.

Most ALMs fail to ground their audio deepfake detection rationales in actual acoustic evidence, despite generating plausible explanations.

Brain-to-text decoding performance gains were an artifact of timing shortcuts, not neural signals, and removing this artifact unlocks substantial improvements with simple methods.

AI-driven music comparison tools can effectively surface subtle, expert-validated differences by letting users precisely scope comparisons and inspect AI-interpreted features.

Immersive AR/VR experiences can finally have stereo audio that dynamically tracks visual sources, thanks to a novel framework that binds visual motion to sound localization.

Emotion vectors are not monolithic directions: decoupling the generic shift away from neutrality from the emotion-specific residual more than doubles training-free steering precision in frozen TTS backbones.

Speech SSL works better without the baggage of discrete codebooks or EMA teachers: predicting raw latent states under Gaussian regularization outpaces existing sub-90M baselines by 43% on frozen ASR.

A subtle "question-relay" mechanism, not direct generation pathways, is the primary driver of AVLLM hallucinations, and steering these question states can effectively mitigate them.

Key contribution not extracted.

Multi-round active simulation turns standard neural density estimators into high-precision, test-time-specialized solvers for notoriously ill-posed physical inverse problems.

SSL-based audio deepfake detectors consistently fail on out-of-distribution attacks because static layer aggregation blinds them to localized spoofing artifacts—a vulnerability resolved here through dynamic, sample-adaptive layer gating and multi-granularity feature fusion.

Contemporary frontier speech models still catastrophically fail on living Arabic dialects, with top systems like GPT-4o achieving an error rate no better than 35.0% WER across 17 regional vernaculars.

Standard speech metrics fail to capture how code-switching transcription errors cascade into critical task failures in production enterprise voice agents.

Conversational 3D heads no longer need static weights: turning live dyadic audiovisual streams into an online self-supervised training signal makes avatar motion up to 11.1% more coherent as dialogues unfold.