Search papers, labs, and topics across Lattice.
Eureka automates feature engineering by framing it as an agentic code generation problem, where LLMs generate, evaluate, and iteratively improve executable feature programs. It uses a fine-tuned Expert Agent for structured feature design, an LLM Feature Factory for code translation, and a Self-Evolving Alignment Engine with RL for code quality. Eureka outperforms AutoFE and LLM baselines on public benchmarks and improves cloud resource demand prediction at Alibaba Cloud, increasing fulfillment by 16% and reducing migration by 33%.
LLMs can now autonomously engineer features as executable code, outperforming traditional methods and unlocking significant gains in real-world cloud resource optimization.
Effective features are crucial for predictive model performance, but creating them often requires domain expertise, limiting scalability across applications. We define feature engineering as an agentic code generation problem: features are not static data transformations, but executable programs that can be generated, evaluated, and iteratively improved. We present Eureka, an LLM-driven framework with three stages. (1) An Expert Agent, fine-tuned via SFT on domain knowledge, produces structured feature design plans in JSON format. (2) An LLM Feature Factory translates each plan into executable Python code through chain-of-thought reasoning, turning feature hypotheses into runnable programs. (3) A Self-Evolving Alignment Engine uses Reinforcement Learning (GRPO) with dual-channel reward (metric-based utility + semantic alignment) to enhance code quality. By expressing features as programs, the learned generation patterns can transfer across domains. Evaluated on 7 public benchmarks in healthcare, finance, and social domains, Eureka consistently outperforms both traditional AutoFE and LLM-based baselines. We further demonstrate Eureka's effectiveness on cloud GPU resource demand prediction at Alibaba Cloud, where Eureka improves demand fulfillment rate by 16% and lowers computing resource migration rates by 33%.
DAMO