Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper highlights the critical role of query design in information extraction, demonstrating that optimizing questions can lead to an impressive 18.6 F1-score point increase across various clinical benchmarks, surpassing the performance gains from simply scaling up extraction models. The authors introduce the List of Questions (LoQ) framework, which generates tailored question sets for specific documents, and the FeedQ method, which refines these questions through feedback from extraction outcomes. By fine-tuning lightweight models with the optimized questions, they achieve performance levels comparable to expert-designed baselines while significantly outperforming larger, untuned models.
Optimizing question design can boost information extraction performance more than scaling up model size, with a staggering 18.6 F1-score point improvement.
When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.