Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces LC-ICL, a novel few-shot learning technique that integrates both correct and incorrect examples for in-context learning in information extraction tasks, specifically named entity recognition and relation extraction. By utilizing error-cause labels to annotate negative samples, the method enhances LLMs' understanding of failure modes, allowing them to avoid similar errors during inference. Experimental results demonstrate that LC-ICL significantly outperforms existing few-shot methods, achieving notable performance improvements across various datasets and tasks.
Incorporating negative examples into in-context learning boosts LLM performance in information extraction by revealing the underlying causes of errors.
There has been increasing interest in exploring the capabilities of advanced large language models (LLMs) in the field of information extraction (IE), specifically focusing on tasks related to named entity recognition (NER) and relation extraction (RE).Although researchers are exploring the use of few-shot information extraction through in-context learning with LLMs, they tend to focus only on using correct or positive examples for demonstration, neglecting the potential value of incorporating incorrect or negative examples into the learning process.In this paper, we present LC-ICL a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. This approach enhances the ability of LLMs to extract entities and relations by combining positive samples with negative samples annotated by error-cause labels. These labels expose more detailed error features in erroneous examples, enabling the model to understand why similar predictions fail and avoid repeating such errors during inference.Specifically, our proposed method taps into the inherent contextual information and valuable information in hard negative samples and the nearest positive neighbors to the test and then applies the in-context learning demonstrations based on LLMs. Our experiments on various datasets indicate that LC-ICL outperforms previous few-shot in-context learning methods, delivering substantial enhancements in performance across a broad spectrum of related tasks. These improvements are noteworthy, showcasing the versatility of our approach in diverse scenarios.