Search papers, labs, and topics across Lattice.
Legato 2 introduces a novel pipeline for optical music recognition (OMR) that sequentially extracts symbolic notation and semantic knowledge from sheet music images by processing them system-by-system. This approach allows for better handling of longer inputs and incorporates the ability to generate symbolic transcriptions that include embedded textual content, such as titles and annotations. The model outperforms previous state-of-the-art methods across multiple datasets and enhances the performance of frontier language models in interpreting complex musical documents.
Legato 2 sets a new benchmark in optical music recognition by seamlessly integrating symbolic transcriptions with visual inputs, revolutionizing how we understand sheet music.
We propose a novel pipeline, Legato 2, for extracting symbolic notation and semantic knowledge from images of sheet music. Legato 2 features the first large-scale neural model for optical music recognition (OMR) to operate sequentially on a system-by-system basis, following the horizontal lines of notation as they are read on the page, rather than treating the page as an undifferentiated image, enabling better scaling to arbitrarily long inputs. It is also the first OMR model capable of generating symbolic transcriptions that include embedded textual content, such as titles and annotations. The pipeline combines system-level segmentation with an autoregressive vision-LM to capture both local notation details and score structure. Across multiple datasets, Legato 2 consistently outperforms prior state of the art. We also show that symbolic transcriptions complement visual inputs for frontier language models, improving their interpretation of dense musical documents. Legato 2 establishes new state-of-the-art performance in both OMR and downstream sheet music understanding.
AI2