Search papers, labs, and topics across Lattice.
This study investigates the capability of large language models (LLMs) to analyze time series data, a critical task in fields like finance and healthcare. By comparing three approaches鈥攔aw numerical data input, coding agent functionality, and a hybrid model鈥攖he authors find that coding agents can outperform those using raw data alone by up to 10%. Despite this improvement, the best coding agents still struggle with a significant portion of questions, indicating persistent reasoning gaps and highlighting the need for better model training in time series analysis.
Coding agents can outperform raw data models in time series analysis, but still miss 22-34% of questions, revealing critical reasoning gaps.
Large language models (LLMs) are increasingly being used for automated decision-making systems in finance, healthcare, or environmental monitoring. Time series data are ubiquitous in these fields, yet hard to process automatically. Can time series be analyzed by LLM agents? We examine three approaches: providing the agent with raw numerical data, using the LLM as a coding agent, or a combination of both. In the coding agent setup, the model iteratively queries the data using Python code. Using two time series understanding benchmarks, we show that agents with code access can outperform models processing raw data by up to 10%. However, even the best performing agent still answers about 22-34% of the questions incorrectly. To get insights into models' strategies and reasoning gaps, we analyze the model outputs with a strong LLM judge. Our analysis reveals that coding agents can select appropriate statistical tests, but often miss important nuances. Meanwhile, models with access to raw data can reach the right conclusions using back-of-the-envelope calculations.
