Search papers, labs, and topics across Lattice.
This paper investigates the robustness of zero-shot relation extraction models under realistic assumptions, specifically addressing the limitations of prior work that relies on pre-encoded mention pairs and lacks a rejection mechanism for irrelevant inputs. The authors categorize existing models, propose strategies for single-pass extraction and incorporating rejection, and adapt state-of-the-art tools to a more challenging evaluation setting. Their experiments reveal that current models struggle with realistic scenarios, although AlignRE demonstrates the best overall performance.
Existing zero-shot relation extraction models crumble when faced with realistic data scenarios, highlighting a critical gap between research and real-world applicability.
Zero-shot relation extraction aims to identify relations between entity mentions using textual descriptions of novel types (i.e., previously unseen) instead of labeled training examples. Previous works often rely on unrealistic assumptions: (1) pairs of mentions are often encoded directly in the input, which prevents offline pre-computation for large scale document database querying; (2) no rejection mechanism is introduced, biasing the evaluation when using these models in a retrieval scenario where some (and often most) inputs are irrelevant and must be ignored. In this work, we study the robustness of existing zero-shot relation extraction models when adapting them to a realistic extraction scenario. To this end, we introduce a typology of existing models, and propose several strategies to build single pass models and models with a rejection mechanism. We adapt several state-of-the-art tools, and compare them in this challenging setting, showing that no existing work is really robust to realistic assumptions, but overall AlignRE (Li et al., 2024) performs best along all criteria.
