Speech & Audio - Weekly Roundup

Decoy Direction Optimization (DDO), a fast, post-hoc weight-editing defense that requires no base-model finetuning, is introduced, based on a simple mechanistic insight: ablation attacks rely on contrastive estimators to find the refusal direction.

Results show that targeted preprocessing, speaker selection, and domain-aware post-processing can substantially improve agricultural speech transcription while preserving the underlying ASR model.

Decoupled-Live-ProsodyJudge (D-LPJ), a cost-effective pairwise evaluator distilled from Gemini into Qwen3-Omni, and a novel span-local GRPO strategy that applies normalized advantages strictly to their corresponding rationale spans.

A discriminative adaptation that reads the final prompt token's hidden state through a classification head, producing a label in one forward pass without modifying the backbone, gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM.

Results show that targeted preprocessing, speaker selection, and domain-aware post-processing can substantially improve agricultural speech transcription while preserving the underlying ASR model.

Decoupled-Live-ProsodyJudge (D-LPJ), a cost-effective pairwise evaluator distilled from Gemini into Qwen3-Omni, and a novel span-local GRPO strategy that applies normalized advantages strictly to their corresponding rationale spans.

A discriminative adaptation that reads the final prompt token's hidden state through a classification head, producing a label in one forward pass without modifying the backbone, gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM.

V\={a}kQA, a Telugu SQA benchmark of 2,001 factoid question-answer pairs across six domains, is introduced and it is observed that Telugu phrasing retains cultural specificity that is lost in translation, speech input introduces phonetic confusions that alter question meaning, and cascaded ASR-MT errors compound progressively.

Age, digital literacy, and interaction modality as correlates of psychomotor and cognitive-flexibility performance in immersive virtual reality (VR) are examined and age was associated with slower task times across modes, but controller direct touch carried the steepest relative age gradient.

Adaptive Score-based Routing Balancer is introduced, a dynamic, score-based request routing mechanism for Kubernetes-based service deployments over the computing continuum that jointly considers infrastructure-level information, response time measurements, and application-level quality indicators, with a particular focus on serving Machine Learning workloads.

The analysis separates the contributions of initial conditions, fast-timescale tracking error, Markovian dependence, and timescale coupling, thereby clarifying the origin of the $\beta^2/\alpha^2$ term.

The Multi-Turn Voice Agent Benchmark (MTVA-Bench), which evaluates the language model on the same conditions it faces inside a cascaded system, and contains 49 agents working across 490 reviewed scenarios and supports 7 languages.

This paper presents a context-aware neural G2P method that scores paths of a discriminative conditional random field over a word lattice constructed from dictionaries that strongly outperforms conventional morphological analyzer-based methods and neural sequence models.

This work presents a multimodal conversational-context framework for LLM-based ASR that integrates a scenario-controlled data pipeline, scalable multimodal context training, and systematic evaluation and introduces MM-ContextASR Bench, which evaluates contextual understanding and entity error correction across five scenarios.

An alignment-path distillation framework for streaming automatic speech recognition (ASR) with large language models (LLMs) that extracts monotonic alignment paths from a non-streaming teacher's soft text-audio attention and uses them to construct interleaved training sequences.

This paper introduces PersianVox, a fully automated pipeline designed to generate high-quality speech corpora from unlabeled web data, using a novel prosody-aware segmentation strategy that utilizes acoustic turn-detection to preserve linguistic completeness and optimize utterance duration for long-context modeling.

HearInContext, a Mandarin--English benchmark that pairs shared synthetic speech with assistant replies supporting different interpretations, is introduced, a Mandarin--English benchmark that pairs shared synthetic speech with assistant replies supporting different interpretations.

DUPAR is proposed, a conversational retrieval framework with complementary slow and fast paths that significantly reduces retrieval errors when the previous turn retrieves correct evidence and the follow-up targets a one-hop neighboring chunk.

Backend delegation is an effective and modular approach for combining natural duplex speech interaction with strong agentic tool-call capabilities and significantly outperforms GPT-realtime-mini and Qwen3-Omni-30B-A3B-Instruct on EVA-Bench.

AUDITPLAN is a single-model plan-then-answer approach where the model first emits a compact structured safety plan and then answers conditioned on it, enabling machine-checkable auditing while remaining hidden from users at deployment.

A multi-teacher knowledge distillation framework to train a lightweight, streaming content encoder that generalizes across healthy (HE) and EL speech that reduces the EL word error rate to 21.2%, compared to 39.3% for the strongest zero-shot SSL baseline.

AWED, a word-level emission-delay metric defined relative to the acoustic end of each word, is introduced and post-train a DSM recognizer with GRPO under a reward that scores transcription accuracy and measured delay jointly, advancing the accuracy--latency Pareto frontier of streaming ASR without architectural change.

This work presents Encypher, a collaborative generative music system that translates collective movement qualities into text prompts conditioning real-time music generation for dance cyphers, and offers a framework and design implications for AI systems for collaborative, embodied expression.

Encoder Awakening via Adapters (EAVA), a simple yet effective domain-adaptive fine-tuning method for Speech-LLM-based ASR, consistently outperforms vanilla fine-tuning and other baselines, achieving new state-of-the-art performance.

There is an urgent need to develop a standardized evaluation infrastructure to assess whether ambient clinical scribes systems are safe, reliable, and well-suited to the Indian healthcare setting, and the development of a publicly shared, real-world, multilingual benchmark for ACS evaluation is called for.

RoleBreak, an open benchmark for long-horizon role-playing robustness in spoken dialogue, is introduced and four key patterns are found, highlighting persistent gaps in both long-horizon robustness and vocal expressiveness in spoken role-playing systems.

This technical retrospective offers a different account of the CosyVoice lineage, from CosyVoice through CosyVoice 2 and CosyVoice 3 to Qwen-Audio-3.0-TTS, and formalizes this history through four interface dimensions---representation, ownership, availability, and gradient reach---and separate within-paper evidence from cross-paper comparison.

Localized Attention-Constrained Inference (LACI), an inference-only method to detect TTS errors in near real-time, roll back to the error onset and regenerate with temporary guardrails, adding negligible computational overhead to demonstrate the efficacy of LACI on voice cloning reliability.

A frozen backbone reads a sentence containing a common word it already says correctly, and its own output then serves as the teacher for the same sentence, with that word replaced by a tagged, accented reading; this training pair is the whole idea.

LACE (Layer-Adaptive Codec Encoding), a dynamic frame rate codec that applies an independent compression step at each quantization layer, enabling layer-specific segmentation boundaries, is proposed.

Experiments conducted on the UASpeech and TORGO corpora suggest that Conformer models trained using the DSI tokens outperform the comparable baseline HuBERT discrete/continuous features by statistically significant WER reductions.

Decoy Direction Optimization (DDO), a fast, post-hoc weight-editing defense that requires no base-model finetuning, is introduced, based on a simple mechanistic insight: ablation attacks rely on contrastive estimators to find the refusal direction.

Comparing matched records of the same cases under fixed labels and splits in three systems showed different patterns: the NHTSA defect summary remained strongest under every model family tested, whereas the ASRS analyst synopsis outperformed the reporter narrative under learned sequence models but not under lexical baselines.

This method integrates the LMs directly into the parameters of an LLM-based ASR model, requiring no additional computational cost at inference, and consistently improved the ASR performance in the target domains, without degrading inference speed or memory footprint.

This work investigates parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper, and proposes Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter.

A deployable system for streaming Thai ASR that lets a user steer its vocabulary at decode time, without retraining is presented, which lifts keyword recall from 16.6% to 20.7% at no accuracy cost and negligible overhead.

This paper presents Cross-Lingual F5-TTS 2, a simplified framework for transcript-free cross-lingual voice cloning without forced alignment, and makes the syllable-level speaking rate predictor robust to leading and trailing silence through silence-aware augmentation.

This paper presents models that add timestamps and speaker information to a given transcript using a non-autoregressive LLM-based architecture and achieves state-of-the-art timestamp accuracy and the best cpWER for speaker attribution.