Search papers, labs, and topics across Lattice.
This paper introduces MusicLayout, an explicit intermediate representation designed to enhance controllability in text-to-music generation by structuring musical elements such as sections and instrument arrangements. By integrating MusicLayout into an autoregressive framework, the authors enable users to inspect and modify the musical layout before audio generation, which significantly improves the structural organization of the resulting compositions. Evaluation results demonstrate that this approach not only facilitates better long-range structural coherence but also empowers users with layout-level control over the generated music.
Explicitly planning musical structure with MusicLayout allows for unprecedented control and inspection in text-to-music generation, transforming how we create and refine compositions.
Text-to-music generation has advanced rapidly, but current systems still rely primarily on global text prompts, leaving the structural organization of generated music implicit and difficult to inspect, control, or revise before audio generation. To address this issue, we introduce MusicLayout, an explicit intermediate representation for controlling musical structure in text-to-music generation. MusicLayout describes a musical piece as a time-aligned layout of sections, textures, repetitions, variations, and instrument-level arrangements, serving as an interpretable planning layer between textual intent and the generated music. We integrate MusicLayout into a text-to-music framework built on a unified autoregressive formulation, where the model first generates a MusicLayout representation and subsequently predicts audio tokens conditioned on this representation within a single sequence. The resulting MusicLayout can be inspected and modified prior to audio generation, providing a mechanism for layout-level structural control. We evaluate MusicLayout through layout-conditioned generation, layout manipulation experiments, and matched-data ablations, providing evidence that explicit layout planning can improve long-range structural organization and support layout-level control.
