Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces CONSISTRE, a unified framework that enhances document-level relation extraction (DocRE) by ensuring consistency across predicted relational triples. By employing a dual approach that combines constraint-aware prompting and knowledge distillation with reinforcement learning, the framework effectively refines predictions from black-box large language models (LLMs) and smaller open-source models. Experimental results on the DocRED dataset demonstrate that CONSISTRE significantly improves extraction performance and reduces relational contradictions, achieving competitive F1 scores without the need for extensive fine-tuning.
CONSISTRE bridges the gap between powerful LLMs and smaller models, ensuring relational consistency and significantly improving extraction reliability.
Document-level relation extraction (DocRE) aims to extract relations among multiple entities across extended contexts while maintaining consistency across predicted triples. Although large language models (LLMs) show remarkable reasoning capabilities in information extraction, their predictions are typically generated independently for each candidate triple and may violate fundamental relational constraints such as transitivity, symmetry, and functional uniqueness, leading to contradictory and unreliable outputs. We propose CONSISTRE, a unified consistency-aware framework for DocRE that addresses this limitation through two complementary tracks. The first operates at inference time for black-box LLMs, combining constraint-aware prompting, constraint-based verification, and iterative self-reflection to refine predictions without task-specific fine-tuning. The second injects consistency knowledge into smaller open-source models via a knowledge distillation and reinforcement learning pipeline: reasoning traces from a powerful teacher are distilled into a student via supervised fine-tuning, followed by GRPO alignment using a composite reward that jointly optimizes extraction performance and relational consistency. Together, the two tracks cover both API-accessible and locally deployable scenarios under a unified consistency formulation. Experiments on DocRED show that both tracks outperform their baselines, with the inference-time track achieving competitive F1 using off-the-shelf black-box LLMs and the training-time track substantially narrowing the gap between 7--8B open-source models and state-of-the-art proprietary LLMs at a fraction of their inference cost. Ablation studies confirm that explicit consistency modeling mitigates relational contradictions and enhances the reliability of LLM-based DocRE across both deployment paradigms.