Search papers, labs, and topics across Lattice.
This paper introduces Bar-JEPA, a novel pipeline for extracting numerical values from bar charts using a joint-embedding predictive architecture (JEPA) that operates in a self-supervised manner. By leveraging high-quality semantic features learned from unlabeled data, the model significantly reduces the reliance on scarce annotated datasets while outperforming traditional supervised methods in value recovery tasks. The results demonstrate that the JEPA encoder effectively enhances the quality of extracted features, leading to improved performance in bar chart data extraction.
Self-supervised learning can unlock high-quality data extraction from bar charts without the need for extensive labeled datasets.
Bar charts are commonly used in data visualization, and while they are easily understood by humans, it is non-trivial to extract the underlying data computationally. For a machine-learning-based approach, training chart de-rendering models usually requires labeled, real-world data. Labeling data is a time consuming task, which is why annotated data is scarce. Models can learn more efficiently when provided with features of high semantic quality, which a joint-embedding predictive architecture (JEPA) is designed to learn in a self-supervised manner. We present a per-bar, numerical value recovery pipeline for bar charts, where a JEPA encoder is used to produce semantically rich latent features. The decoder model consuming these features is simple and quick to train and outputs the coordinates of ticks and bars, which can be used to recover bar values. The effectiveness of self-supervised finetuning and quality of the extracted features is evident when comparing our model to end-to-end supervised baselines. Code, datasets and checkpoints are available on \href{https://github.com/dralois/Bar-JEPA}{GitHub}.
