Search papers, labs, and topics across Lattice.
This paper tackles the intricate challenge of understanding the hierarchical structures of newspaper images by introducing two complementary approaches: a modular bottom-up pipeline utilizing existing models for layout detection and reading order prediction, and Tiramisu, a novel transformer-based architecture designed for hierarchical structure understanding. The modular approach combines YOLO, LayoutReader, and a custom segmentation algorithm, while Tiramisu employs tiered transformers to perform tasks such as section separation and semantic categorization through parallelized attention mechanisms. Experimental results reveal that both methods effectively reconstruct complex newspaper hierarchies, providing valuable insights for scalable document digitization, alongside the release of a new dataset for evaluating these techniques.
Tiramisu's tiered transformer architecture revolutionizes newspaper image understanding by explicitly modeling document hierarchy, outperforming traditional methods in reconstructing complex layouts.
Understanding newspaper images remains a challenging task due to their complex, nested hierarchical structures and dense, heterogeneous layouts. In this paper, we explore two complementary approaches for newspaper structure understanding. First, we present a modular bottom-up pipeline that combines state-of-the-art open-source models: YOLO for layout detection, LayoutReader for reading order prediction, and a custom algorithm for article segmentation. This approach leverages existing robust components while maintaining flexibility and interpretability. Second, we introduce Tiramisu (Tiered Transformers for Hierarchical Structure Understanding), a novel end-to-end transformer-based architecture that explicitly models document hierarchy through an iterative tiered process. Tiramisu performs section and article separation, block localization, semantic categorization, and reading order prediction using highly parallelized attention mechanisms. Finally, we release Finlam La Libert\'e, a new dataset designed specifically for evaluating hierarchical information retrieval in historical newspapers. Experimental results demonstrate the effectiveness of both approaches in reconstructing complex newspaper hierarchies, with comparative analysis highlighting their respective strengths for scalable document digitization. The Tiramisu training code, including the synthetic newspaper generator, is available at https://git.litislab.fr/tiramisu/tiramisu-newspaper-articles-extractor.
