Search papers, labs, and topics across Lattice.
This paper introduces Harness TTS, a novel control layer for text-to-speech (TTS) systems that enhances expressive speech synthesis by enabling context-aware style adaptation. By employing a lightweight architecture that utilizes an LLM planner for prompt-tool routing, the system achieves significant improvements in both routing accuracy and synthesis quality, outperforming traditional instruction-only methods. Notably, the 4B planner provides tool recommendations with minimal latency, making it suitable for real-time voice assistant applications.
Harness TTS achieves up to 35.6-point improvements in instruction-following win rates, revolutionizing expressive speech synthesis for voice assistants.
Expressive speech synthesis for voice assistants requires flexible style control that adapts to explicit requests and broader interaction context. We propose Harness TTS, a lightweight control layer that wraps around a TTS engine to externalize and govern its expressive behavior. It reformulates style control as closed-set prompt-tool routing: offline, a compact registry of stylistic prompt tools is constructed with structured metadata; online, an LLM planner selects the appropriate tool based on a priority-aware observation schema, and the TTS executor synthesizes speech using the corresponding prompt audio. We evaluate Harness TTS on both routing and synthesis tasks. In routing, Qwen3-4B achieves Top-1 accuracies of 74.3%, 43.0%, and 64.6% on explicit, implicit, and conflict subsets. For synthesis, experiments on CosyVoice3 and VoxCPM2 show that Harness TTS outperforms instruction-only control, achieving higher instruction-following win rates (margins of 23.1-35.6 points on CosyVoice3 and 13.8-20.0 points on VoxCPM2) and improving UTMOSv2 scores by 0.11-0.38. Moreover, the 4B planner delivers its first tool recommendation in under 50 ms in standard mode, introducing negligible latency for real-time interaction. These results demonstrate that equipping TTS engines with a dedicated Harness layer offers a practical, auditable, and context-aware solution for voice assistant expression control.
