Speech & Audio - Weekly Roundup

Explicitly aligning video captions with atomic audiovisual events unlocks substantial gains in fine-grained temporal and attributional accuracy, outperforming unstructured generation by over 9% on event-centric metrics.

Audio embeddings capture physical damping properties far better than single visual frames, and this gap is partially bridgeable through audio-to-vision distillation.

Neural audio watermarking detectors can now reliably detect legacy content even after updates, while simultaneously rejecting unauthorized embedders, by explicitly training for angular separation in the embedding space.

Brain activity can decode naturalistic speech segments with 42% accuracy within the top 10 of 2,000 options, a feat achieved by aligning fMRI with multimodal stimulus embeddings.

A neural operator reconstructs continuous HRTF fields from sparse measurements with 18% lower LSD and 43% lower ILD error than leading baselines, achieving 373x speedup over neural fields.

Explicitly aligning video captions with atomic audiovisual events unlocks substantial gains in fine-grained temporal and attributional accuracy, outperforming unstructured generation by over 9% on event-centric metrics.

Audio embeddings capture physical damping properties far better than single visual frames, and this gap is partially bridgeable through audio-to-vision distillation.

Neural audio watermarking detectors can now reliably detect legacy content even after updates, while simultaneously rejecting unauthorized embedders, by explicitly training for angular separation in the embedding space.

Brain activity can decode naturalistic speech segments with 42% accuracy within the top 10 of 2,000 options, a feat achieved by aligning fMRI with multimodal stimulus embeddings.

A neural operator reconstructs continuous HRTF fields from sparse measurements with 18% lower LSD and 43% lower ILD error than leading baselines, achieving 373x speedup over neural fields.

One-step RF signal separation achieves superior BER and MSE by learning a direct clean-endpoint transport via rectified flow matching.

A single open-source model now performs multi-character scene dubbing, in-filling edits, and multi-turn revisions with natural language direction, all at 271 fps.

A single adversarial token can bypass most safety filters.

Emotions' hierarchical structure is finally captured in multimodal recognition by fusing Euclidean token-level fusion with hyperbolic core routing, achieving state-of-the-art results.

Physical controls like damping and size can precisely guide audio foley generation, achieving near-perfect correlation with real-world sound targets without fine-tuning.