Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
MusiChat introduces a conversational vibe composing system that allows users to iteratively refine musical compositions through natural-language interactions, overcoming the limitations of traditional prompt-and-regenerate approaches. By employing a hierarchical controllable music generation framework, it separates the generation of musical structure from expressive surface realization, enabling flexible edits while preserving the integrity of the composition. Evaluations show that MusiChat achieves high accuracy in user interactions and favorable ratings for melody naturalness and musical quality, highlighting its effectiveness in collaborative music creation.
MusiChat transforms how we compose music by enabling real-time, iterative collaboration with AI, rather than starting from scratch each time.
Recent advances in AI music generation have enabled users to create complete musical pieces from natural-language prompts. However, most existing systems follow a prompt-and-regenerate paradigm, making iterative refinement difficult because users must repeatedly recreate compositions instead of directly evolving existing musical ideas. We present MusiChat, a conversational vibe composing system that enables collaborative human-AI music creation through natural-language interaction and iterative refinement. At the core of MusiChat is a hierarchical controllable music generation framework that separates lyric-aligned musical structure generation from expressive surface realization, allowing flexible stylistic transformations and structure-preserving edits. The system integrates a large language model with a hybrid symbolic music engine through a memory-augmented architecture that maintains the active composition state and user history across interactions. A hybrid intent-routing mechanism further enables efficient interpretation of both precise musical edits and open-ended creative requests. Rather than regenerating compositions from scratch, MusiChat incrementally transforms an evolving musical artifact while preserving relevant musical structure and user intent. We evaluate MusiChat through objective analysis and human studies, achieving 95.31% and 100% accuracy for single- and multi-turn interactions, respectively, and obtaining like-to-dislike ratios of 2:1 for melody naturalness and 3:1 for musical quality. Our results demonstrate that MusiChat supports coherent multi-turn music authoring and interactive human-AI co-creation through a conversational interface.