Speech & Audio

Explicitly aligning video captions with atomic audiovisual events unlocks substantial gains in fine-grained temporal and attributional accuracy, outperforming unstructured generation by over 9% on event-centric metrics.

A novel event-centric evaluation framework, AVC-Reward, finally provides a nuanced assessment of audio-video generation beyond holistic metrics, revealing fine-grained cross-modal failures.

Audio embeddings capture physical damping properties far better than single visual frames, and this gap is partially bridgeable through audio-to-vision distillation.

Neural audio watermarking detectors can now reliably detect legacy content even after updates, while simultaneously rejecting unauthorized embedders, by explicitly training for angular separation in the embedding space.

Brain activity can decode naturalistic speech segments with 42% accuracy within the top 10 of 2,000 options, a feat achieved by aligning fMRI with multimodal stimulus embeddings.

Explicitly modeling metrical structure alongside note events halves transcription error in expressive piano performances, bridging the gap between raw audio and symbolic notation.

A neural operator reconstructs continuous HRTF fields from sparse measurements with 18% lower LSD and 43% lower ILD error than leading baselines, achieving 373x speedup over neural fields.

One-step RF signal separation achieves superior BER and MSE by learning a direct clean-endpoint transport via rectified flow matching.

A single open-source model now performs multi-character scene dubbing, in-filling edits, and multi-turn revisions with natural language direction, all at 271 fps.

A single adversarial token can bypass most safety filters.

Emotions' hierarchical structure is finally captured in multimodal recognition by fusing Euclidean token-level fusion with hyperbolic core routing, achieving state-of-the-art results.

Physical controls like damping and size can precisely guide audio foley generation, achieving near-perfect correlation with real-world sound targets without fine-tuning.

Beyond transcribing words, a novel convolutional-Transformer model now timestamps non-verbal vocalizations and background sounds, enriching ASR with a richer auditory context.

Self-supervised speech embeddings unlock substantial gains in cross-speaker dysarthria detection during awake craniotomy, outperforming handcrafted features by up to 26.1% AUC.

Frontier MLLMs exhibit strong local perception but falter on long-horizon audio-visual reasoning, with a single benchmark exposing specific failure modes like identity drift and cross-modal misalignment.

Complex-valued fusion of audio-visual embeddings unlocks significant gains in open-vocabulary event localization, surpassing prior methods by up to 6.6% Event-F1.

Decouple complex audio scenes into individual, controllable sound sources directly from a single generative model.

Audio-language models excel at identifying sounds but fail to grasp the crucial context of how speakers react to them.

Fine-grained emotion control in TTS is now achievable at inference time with minimal speaker or content degradation, bypassing the need for expensive model retraining.

Black-box text-to-music generators can be steered to precisely match complex user intent by an agent that iteratively refines prompts based on rubric-based audio verification.

Latency in voice assistants plummets by 20% by speculatively executing tool calls *during* speech recognition, not after.

Interactive AI worlds can now speak and react in real-time with spatially accurate sound, bridging the sensory gap in generative environments.

A single-voice TTS model with 10x fewer parameters and 15x less compute than its teacher achieves near-identical audio quality, runnable on a single laptop.

Wrist accelerometers can now yield population-scale sleep heart rate insights, with a novel transformer model outperforming existing methods in estimation availability.

A novel cross-domain recommendation framework achieves state-of-the-art book-to-music mood matching by fusing sentiment embeddings with LLM-driven emotional quadrant classification.

Simple vMF likelihood modeling unlocks robust label-free speaker embedding enhancement, outperforming complex diffusion baselines on challenging acoustic mismatches.

Iterative refinement of a class token with shared blocks slashes transformer parameters and computation while boosting audio recognition accuracy.

Acoustic noise, even when not diagnostically informative, can systematically manipulate Alzheimer's predictions in SSL models, revealing a critical vulnerability in current speech-based assessments.

Zero-shot voice conversion now achieves $9\times$ faster inference with improved perceptual quality by jointly optimizing flow and content encoding.

Most ALMs fail to ground their audio deepfake detection rationales in actual acoustic evidence, despite generating plausible explanations.

Brain-to-text decoding performance gains were an artifact of timing shortcuts, not neural signals, and removing this artifact unlocks substantial improvements with simple methods.

AI-driven music comparison tools can effectively surface subtle, expert-validated differences by letting users precisely scope comparisons and inspect AI-interpreted features.

Immersive AR/VR experiences can finally have stereo audio that dynamically tracks visual sources, thanks to a novel framework that binds visual motion to sound localization.

Emotion vectors are not monolithic directions: decoupling the generic shift away from neutrality from the emotion-specific residual more than doubles training-free steering precision in frozen TTS backbones.

Speech SSL works better without the baggage of discrete codebooks or EMA teachers: predicting raw latent states under Gaussian regularization outpaces existing sub-90M baselines by 43% on frozen ASR.

A subtle "question-relay" mechanism, not direct generation pathways, is the primary driver of AVLLM hallucinations, and steering these question states can effectively mitigate them.

Key contribution not extracted.

Multi-round active simulation turns standard neural density estimators into high-precision, test-time-specialized solvers for notoriously ill-posed physical inverse problems.

SSL-based audio deepfake detectors consistently fail on out-of-distribution attacks because static layer aggregation blinds them to localized spoofing artifacts—a vulnerability resolved here through dynamic, sample-adaptive layer gating and multi-granularity feature fusion.

Contemporary frontier speech models still catastrophically fail on living Arabic dialects, with top systems like GPT-4o achieving an error rate no better than 35.0% WER across 17 regional vernaculars.

Standard speech metrics fail to capture how code-switching transcription errors cascade into critical task failures in production enterprise voice agents.

Conversational 3D heads no longer need static weights: turning live dyadic audiovisual streams into an online self-supervised training signal makes avatar motion up to 11.1% more coherent as dialogues unfold.

Control accent strength in zero-shot TTS with a single model and inference-time weight, outperforming cascaded systems on out-of-domain accents without sacrificing speaker identity.