Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Just Pass Twice (JPT), a method for zero-shot Named Entity Recognition (NER) that leverages causal LLMs for discriminative token classification with bidirectional context. JPT concatenates the input sentence to itself, enabling each token in the second pass to attend to the entire sentence without architectural modifications, and combines these representations with definition-guided entity embeddings. JPT achieves state-of-the-art zero-shot NER results, outperforming previous methods by +7.9 F1 on average and achieving a 20x speedup compared to generative approaches.
LLMs can achieve state-of-the-art zero-shot NER with a simple "Just Pass Twice" trick that unlocks bidirectional context, bypassing the need for slow, hallucination-prone generative methods.
Large language models encode extensive world knowledge valuable for zero-shot named entity recognition. However, their causal attention mechanism, where tokens attend only to preceding context, prevents effective token classification when disambiguation requires future context. Existing approaches use LLMs generatively, prompting them to list entities or produce structured outputs, but suffer from slow autoregressive decoding, hallucinated entities, and formatting errors. We propose Just Pass Twice (JPT), a simple yet effective method that enables causal LLMs to perform discriminative token classification with full bidirectional context. Our key insight is that concatenating the input to itself lets each token in the second pass attend to the complete sentence, requiring no architectural modifications. We combine these representations with definition-guided entity embeddings for flexible zero-shot generalization. Our approach achieves state-of-the-art results on zero-shot NER benchmarks, surpassing the previous best method by +7.9 F1 on average across CrossNER and MIT benchmarks, being over 20x faster than comparable generative methods.