Search papers, labs, and topics across Lattice.
This paper investigates the limitations of applying GPT-style models to symbolic music by emphasizing the importance of effective tokenization and coordinate systems for predictive compression. It introduces the Effectiveness--Losslessness Framework, which distinguishes between the Fact--Token Boundary and the Token--State Boundary, highlighting how effective tokenization requires a balance between predictively effective representations and relational losslessness. The findings reveal that while sequence compaction is beneficial, it does not ensure predictive compression, and maintaining relational freedom is crucial for higher-order musical organization to develop without explicit structural labels.
Tokenization in music must prioritize predictive effectiveness over mere sequence compaction to unlock higher-order musical structures.
GPT-style models achieve strong performance by representing language with finite vocabularies of reusable discrete tokens. This success has motivated symbolic music tokenizations to treat recurring musical structures, such as chords, motifs, and phrases, as reusable units analogous to linguistic tokens. However, tokenization derives its advantage not from reusable combinations alone, but from compression: effective compression requires coordinates in which recurring regularities form stable and predictable conditional distributions. The key problem is therefore not to find larger musical combinations, but to discover the coordinate system in which musical facts become predictively compressible. We formulate the Effectiveness--Losslessness Framework and define tokenization as the construction of a predictively effective and relationally lossless coordinate system. The Predictive Effectiveness Principle defines the Fact--Token Boundary: decoupling and denesting construct coordinate interfaces that expose predictive regularities. The Relational Losslessness Principle defines the Token--State Boundary: tokenization stops before context-dependent relations are fixed, leaving their computation to model states. Controlled symbolic-music experiments validate these boundaries. Effective coordinate construction improves predictive compressibility, while fixed relational projections constrain contextual modeling. Sequence compaction alone does not guarantee predictive compression, while preserving contextual freedom allows higher-order musical organization to emerge without explicit structural labels. These results reveal why GPT-style models do not transfer directly across modalities: architectures transfer, but tokenization interfaces do not. Tokenization must discover effective representations while preserving the relational freedom from which contextual structure can emerge.
