Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses temporal domain generalization (TDG) for LLMs by reformulating it geometrically under parameter-efficient fine-tuning. It posits that the low-dimensional temporal structure of model evolution can be preserved under parameter-efficient reparameterization. The authors introduce Manifold-aware Temporal LoRA (MaT-LoRA), which constrains temporal updates to a shared low-dimensional manifold within a low-rank adaptation subspace, modeling its evolution through a structured temporal core, and achieving superior temporal generalization performance with practical scalability.
LLMs can adapt to evolving data distributions with dramatically reduced computational cost by constraining temporal updates to a low-dimensional manifold within a LoRA subspace.
Temporal distribution shifts are pervasive in real-world deployments of Large Language Models (LLMs), where data evolves continuously over time. While Temporal Domain Generalization (TDG) seeks to model such structured evolution, existing approaches characterize model adaptation in the full parameter space. This formulation becomes computationally infeasible for modern LLMs. This paper introduces a geometric reformulation of TDG under parameter-efficient fine-tuning. We establish that the low-dimensional temporal structure underlying model evolution can be preserved under parameter-efficient reparameterization, enabling temporal modeling without operating in the ambient parameter space. Building on this principle, we propose Manifold-aware Temporal LoRA (MaT-LoRA), which constrains temporal updates to a shared low-dimensional manifold within a low-rank adaptation subspace, and models its evolution through a structured temporal core. This reparameterization dramatically reduces temporal modeling complexity while retaining expressive power. Extensive experiments on synthetic and real-world datasets, including scientific documents, news publishers, and review ratings, demonstrate that MaT-LoRA achieves superior temporal generalization performance with practical scalability for LLMs.