Search papers, labs, and topics across Lattice.
This paper introduces VTBench, a multimodal framework for time-series classification that fuses raw numerical inputs with chart-based visualizations (line, area, bar, scatter). They systematically evaluate different chart types, visual encoding choices, and fusion strategies across 31 UCR datasets, demonstrating the potential of chart-only models, the benefits of combining multiple chart types, and the impact of redundancy in multimodal fusion. The results provide practical guidelines for selecting chart types and fusion strategies for improved time-series classification.
Time-series classification gets a visual upgrade: fusing raw data with intuitive charts like line, bar, and scatter plots can boost accuracy, especially on smaller datasets.
Time-series classification (TSC) has advanced significantly with deep learning, yet most models rely solely on raw numerical inputs, overlooking alternative representations. While texture-based encodings such as Gramian Angular Fields (GAF) and Recurrence Plots (RP) convert time series into 2D images, they often require heavy preprocessing and yield less intuitive representations. In contrast, chart-based visualizations offer more interpretable alternatives and show promise in specific domains; however, their effectiveness remains underexplored, with limited systematic evaluation across chart types, visual encoding choices, and datasets. In this work, we introduce VTBench, a systematic and extensible framework that re-examines TSC through multimodal fusion of raw sequences and chart-based visualizations. VTBench generates lightweight, human-interpretable plots -- line, area, bar, and scatter, providing complementary views of the same signal. We develop a modular architecture supporting multiple fusion strategies, including single-chart visual-numerical fusion, multi-chart visual fusion, and full multimodal fusion with raw inputs. Through experiments across 31 UCR datasets, we show that: (1) chart-only models are competitive in selected settings, particularly on smaller datasets; (2) combining multiple chart types can improve accuracy by capturing complementary visual cues; and (3) multimodal models improve or maintain performance when visual features provide non-redundant information, but may degrade accuracy when they introduce redundancy. We further distill practical guidelines for selecting chart types, fusion strategies, and configurations. VTBench establishes a unified foundation for interpretable and effective multimodal time-series classification.
