Search papers, labs, and topics across Lattice.
This paper introduces LexKairos, a novel benchmark designed to assess the temporal capabilities of large language models (LLMs) within the Chinese legal framework, focusing on statutory temporal knowledge, case temporal modeling, and statute-case temporal reasoning. Through systematic evaluations of eight LLMs across various inference settings, the study reveals that while Gemini-3-Flash performs best overall, significant shortcomings persist in tasks requiring accurate recall of time-sensitive statutory metadata and complex temporal reasoning. These findings underscore the critical need for improved temporal reasoning in legal AI applications, highlighting ongoing challenges in the field.
Even the top-performing LLM struggles with complex legal temporal reasoning, revealing significant gaps in AI's understanding of time-sensitive legal contexts.
Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that governs the validity of statutes, the progression of legal cases, and the enforcement of procedural deadlines. However, legal temporal capabilities remain underexplored in existing legal AI benchmarks. To address this gap, we propose LexKairos, a comprehensive benchmark for evaluating the temporal capabilities of LLMs in the Chinese legal context across three dimensions: statutory temporal knowledge, case temporal modeling, and statute-case temporal reasoning. LexKairos comprises nine sub-tasks drawn from real-world Chinese judicial cases and statutes. We conduct systematic evaluations of eight LLMs under multiple inference settings, including vanilla, Chain-of-Thought (CoT), and thinking modes. Our results show that Gemini-3-Flash achieves the strongest overall performance, yet even the best-performing model exhibits notable limitations on tasks demanding precise time-sensitive statutory metadata recall or complex reasoning in time limits, indicating that legal temporal knowledge and reasoning remain open challenges for current LLMs. Data and code are available at https://github.com/thunlp/LexKairos.
Tsinghua AI