Search papers, labs, and topics across Lattice.
This paper introduces a rubric-based pipeline that leverages LLMs to analyze input examples and synthesize programmatic specifications for feature extraction, effectively transforming raw text-serialized inputs into standardized formats for downstream models. The approach uses both global rubrics (task-agnostic) and local rubrics (task-conditioned summaries) to improve input representation design. Experiments on 15 clinical tasks from the EHRSHOT benchmark demonstrate that rubric-based approaches significantly outperform traditional count-feature models, naive text-serialization-based LLM baselines, and a clinical foundation model.
Forget hand-engineered features: LLMs can automatically generate rubrics that transform raw text into powerful representations, outperforming even pre-trained clinical models on EHR tasks.
As real-world datasets become increasingly complex and heterogeneous, supervised learning is often bottlenecked by input representation design. Modeling multimodal data for downstream tasks, such as time-series, free text, and structured records, often requires non-trivial domain-specific engineering. We propose an agentic pipeline to streamline this process. First, an LLM analyzes a small but diverse subset of text-serialized input examples in-context to synthesize a global rubric, which acts as a programmatic specification for extracting and organizing evidence. This rubric is then used to transform naive text-serializations of inputs into a more standardized format for downstream models. We also describe local rubrics, which are task-conditioned summaries generated by an LLM. Across 15 clinical tasks from the EHRSHOT benchmark, our rubric-based approaches significantly outperform traditional count-feature models, naive text-serialization-based LLM baselines, and a clinical foundation model, which is pretrained on orders of magnitude more data. Beyond performance, rubrics offer several advantages for operational healthcare settings such as being easy to audit, cost-effectiveness to deploy at scale, and they can be converted to tabular representations that unlock a swath of machine learning techniques.
MIT CSAIL