Search papers, labs, and topics across Lattice.
The paper introduces ViTex, a visual representation of instrumental texture for controlling multi-track symbolic music generation, where color encodes instrument, spatial position represents pitch/time, and stroke captures local textures. They address the lack of direct, intuitive control in existing music generation systems, particularly for multi-track symbolic music. They develop a discrete diffusion model conditioned on ViTex and chord progressions, demonstrating texture-level control while preserving unconditional generation quality for 8-measure compositions.
Control the instrumental texture of your multi-track music compositions with ViTex, a novel visual representation that lets you directly shape the sound using color, position, and stroke properties.
In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.
