Search papers, labs, and topics across Lattice.
RuleChef is a novel framework that leverages large language models to generate and refine executable rules for various NLP tasks such as text classification and Named Entity Recognition (NER). By utilizing task descriptions and labeled examples, RuleChef iteratively enhances these rules through human feedback and additional examples, resulting in a system that is both fast and deterministic. Preliminary evaluations indicate that RuleChef outperforms traditional methods, providing a more transparent and inspectable approach to rule-based NLP systems.
RuleChef transforms LLM-generated task knowledge into human-editable rules, enhancing transparency and adaptability in NLP applications.
We present RuleChef, a framework that uses large language models (LLMs) to generate executable rules for NLP tasks such as text classification, Named Entity Recognition (NER), or relation extraction. Rules are generated based on a task description and a set of labeled examples, then they are iteratively improved based both on additional examples and on human feedback overexisting rules. RuleChef can also be used to bootstrap rules using the observed input-output pairs from any existing model for a given task. LLMs are used only at learning time, synthesizing rules and iteratively patching them based on failures measured on a held-out split. The result of this process is a fast, deterministic, and inspectable rule system. Preliminary evaluation is performed on both classification and NER tasks. We release RuleChef as open-source software under an Apache 2.0
