Speech & Audio

Automatic music transcription models struggle with pop music, as evidenced by a mere 38% Onset F1 score on the new MulTTiPop dataset.

Overlap in conversational training data can significantly reduce ASR error rates, revealing a critical trade-off between overlap and gap timing that reshapes data generation strategies.

State-of-the-art audio encoders may obscure critical frequency-localized features, but a simple post-hoc intervention can recover access and improve interpretability.

MuScriptor achieves unprecedented accuracy in transcribing multi-instrument music, outperforming existing models that rely solely on synthetic data.

Evaluator bias in TTS assessment can lead to misleading quality rankings, with same-family ASR pairs outperforming cross-family pairs by up to 3x.

Reinforcement learning outperforms supervised fine-tuning in adapting ASR systems to synthetic speech, achieving a 40% reduction in word error rates.

Human annotations boost performance, but pseudo-labels unlock scalability in visual speech recognition—what’s the trade-off?

Phoneme-level insights reveal the hidden linguistic cues that differentiate real speech from deepfakes, enhancing trust in detection systems.

Quantization errors in speech enhancement models can be largely mitigated by robust spatial filtering, enabling efficient deployment on low-resource devices.

A multimodal approach to asthma detection reveals that voice recordings can adaptively prioritize features based on symptom severity, achieving an impressive AUROC of 0.85.

COALA outperforms existing methods in contextual biasing for ASR, achieving superior performance even in complex multi-entity scenarios.

Achieving a tcpMER of 17.97, this system reduces error rates significantly by leveraging advanced diarization and ASR adaptation techniques.

Achieving top-tier performance in speaker extraction from real conversations with a novel training framework that leverages proxy supervision and a large-scale dataset.

Eight-bit quantization allows a $1.2$-billion-parameter model to run on an $8$\,GB Raspberry Pi without sacrificing audio quality or speed.

Combining audio and visual analysis in a unified ensemble model boosts deepfake detection accuracy and generalization across diverse manipulations.

SAMPA achieves impressive F1 scores in prosodic boundary detection for Brazilian Portuguese, outperforming traditional methods and revealing the power of fine-tuned Whisper models in this domain.

MM-VAP significantly boosts turn-taking prediction accuracy in social robots by integrating audio-visual cues, outperforming existing models in key conversational scenarios.

A novel weighted distance measure boosts rag classification accuracy by prioritizing key melodic sequences, revealing deeper insights into Tagore's musical identity.

Geometry-consistent generative audio can dramatically enhance spatial learning for visually impaired learners, outperforming traditional audio methods in immersive educational settings.

Current singing voice synthesis models struggle with genre discrimination, showing that genre-specific training can dramatically improve performance where zero-shot methods fail.

Current models struggle to align with human music aesthetic judgments, revealing a substantial gap in understanding that MADB aims to bridge.

Discrete audio tokens can rival traditional spectral features in speaker verification when guided by a robust knowledge distillation framework.

Optimizing conversational timing as a standalone objective can lead to more natural interactions without compromising reasoning abilities in dialogue systems.

UBG-Net outperforms existing models by effectively filtering noise and enhancing robustness in audio-visual speech recognition through advanced uncertainty modeling.

LALM models can match human audio judgment reliability, but not all versions are created equal—3.1 Pro struggles where 3.5 Flash excels.

Gradient-based alignment can outperform traditional methods in challenging scenarios, offering a universal solution for precise speech-to-text mapping across diverse ASR models.

Uncovering the root cause of modality interference, this work achieves a remarkable 28.5% improvement in full-duplex interaction fluidity without sacrificing efficiency.

Flowley not only streamlines video-to-audio generation with a single-stage architecture but also sets a new benchmark for audio quality by leveraging sound-aware captions.

Compressing the KV cache of speech tokens can enhance decoding speed by over 1.49 times while improving performance on key benchmarks.

Distinct training objectives in self-supervised speech models create unique acoustic compression regimes that impact downstream task performance.

ForestIR reveals how precise control over environmental variables can significantly enhance the design and evaluation of bioacoustic monitoring systems in complex forest ecosystems.

A new neural kernel approach captures common directional patterns in sound fields, drastically improving generalization beyond single snapshot measurements.

Achieving the lowest error rates in automatic speech recognition and speaker verification, this method redefines the standards for single-channel speech separation.

CARE-DPP outperforms traditional methods by intelligently balancing uncertainty and novelty, leading to more effective biodiversity classification with less annotation effort.

SR-FD reduces word error rates by over a third, transforming the landscape of intelligibility in few-step TTS synthesis.

Automated benchmarks can now evaluate MLLMs' music perception skills across diverse modalities, ensuring more reliable assessments than ever before.

ORCA not only boosts performance by 26.4 points but also restores critical speaker identity cues that traditional models overlook.

Role-Aware Encoding reveals how nuanced micro-timing in classical music can transform AI's approach to collaborative composition.

Systems with near-perfect EER can still expose significant privacy vulnerabilities, with worst-case disclosures effectively doubling an attacker's success rate.

Achieving high accuracy in audio classification while preventing catastrophic forgetting is now possible with a dual-module system that decouples learning and adaptation.

Automatically generated multilingual transcripts can significantly enhance audio sentiment analysis, leading to improved classification performance.

Achieving nearly 4% accuracy gains in audio classification tasks with a novel automatic annotation pipeline could transform data scarcity challenges in domestic environments.

Harmonizations generated by the proposed system not only preserve tonal integrity but also allow for diverse harmonic interpretations without relying on extensive training data.

Taiwanese Mandarin TTS systems can achieve a 63.9% reduction in word error rates by using a context-adapted tokenizer and language model.

Achieving unprecedented word-level control in TTS, WordVoice allows for fine-tuned acoustic manipulation that transforms audiobook narration and video dubbing.

Continual pre-training on a related language can dramatically enhance ASR performance in low-resource languages like Dhivehi, achieving a 13.50% reduction in WER.

Gender effects in filled pause usage defy expectations, revealing language-specific trends that challenge existing literature.

Students using text prompts outperformed their peers using voice input, highlighting the need for careful consideration of input modalities in programming education.

Naive translation of audio descriptions fails to capture cultural nuances, revealing a critical gap in accessibility for India's Blind and Low Vision communities.

Quantum-inspired methods can transform harmonic decision-making in music, revealing that complexity doesn't always equate to naturalness in composition.