Search papers, labs, and topics across Lattice.
This paper introduces SIGMA, a novel framework for Automated Feature Engineering (AutoFE) that utilizes SHAP values to guide implicit trajectory generation without relying on semantic metadata. By employing an EXposed-feature Implicit Trajectory (EXIT) approach, SIGMA effectively maintains a constant context length while significantly reducing the duplication of generated features from 37.2% to 6.8%. Empirical evaluations reveal that SIGMA achieves performance on par with state-of-the-art LLM baselines while utilizing only 5.4 features on average, highlighting its efficiency in feature generation.
Reducing feature duplication from 37.2% to 6.8% while maintaining state-of-the-art performance with just 5.4 features showcases a breakthrough in metadata-free AutoFE.
Recent research has leveraged Large Language Models (LLMs) to enhance Automated Feature Engineering (AutoFE) through semantic descriptions and trajectory-based prompting. However, there exist two challenges that limit their applicability and scalability in long-horizon optimization: (1) semantic metadata is unavailable in many practical settings, and (2) trajectory accumulation increases the risk of exceeding the context window, while without it, the generation process can become unstable, leading to becoming stuck in the local optima and a high duplicate rate of generated features. To this end, we propose a SHAP-enhanced Implicit-trajectory Generation for Metadata-free AutoFE (SIGMA), a scalable constant-context optimization framework. SIGMA leverages SHAP values to provide task-aware signals for guiding group feature generation instead of semantic information. In addition, we adopt an EXposed-feature Implicit Trajectory (EXIT) approach, where the exposed features in the prompt implicitly represent the trajectory. Empirical results demonstrate that SIGMA achieves performance comparable to the state-of-the-art (SOTA) LLM baselines with a nearly constant prompt length. Notably, EXIT significantly reduces the duplicate ratio of generated features from 37.2% to 6.8%. At the same time, SIGMA matches traditional SOTA performance with only 5.4 features on average, demonstrating substantial efficiency gains in feature utilization.
