🤖 AI Summary
This work addresses the limited generalizability of traditional feature engineering, which heavily relies on domain expertise. The authors propose modeling feature engineering as an agent-driven code generation task: an expert agent first produces a structured feature design plan, which a large language model then translates into executable Python code. A dual-channel reinforcement learning framework based on GRPO continuously refines this process by jointly optimizing feature utility and semantic alignment with the original plan. This approach pioneers representing features as executable programs, integrating structured planning, chain-of-thought reasoning, and knowledge transfer to enable cross-domain automated feature engineering. Evaluated on seven public benchmarks, the method significantly outperforms existing AutoFE and LLM-based approaches. In Alibaba Cloud’s GPU resource demand forecasting task, it improves demand fulfillment rate by 16% and reduces resource migration rate by 33%.
📝 Abstract
Effective features are crucial for predictive model performance, but creating them often requires domain expertise, limiting scalability across applications. We define feature engineering as an agentic code generation problem: features are not static data transformations, but executable programs that can be generated, evaluated, and iteratively improved. We present Eureka, an LLM-driven framework with three stages. (1) An Expert Agent, fine-tuned via SFT on domain knowledge, produces structured feature design plans in JSON format. (2) An LLM Feature Factory translates each plan into executable Python code through chain-of-thought reasoning, turning feature hypotheses into runnable programs. (3) A Self-Evolving Alignment Engine uses Reinforcement Learning (GRPO) with dual-channel reward (metric-based utility + semantic alignment) to enhance code quality. By expressing features as programs, the learned generation patterns can transfer across domains. Evaluated on 7 public benchmarks in healthcare, finance, and social domains, Eureka consistently outperforms both traditional AutoFE and LLM-based baselines. We further demonstrate Eureka's effectiveness on cloud GPU resource demand prediction at Alibaba Cloud, where Eureka improves demand fulfillment rate by 16% and lowers computing resource migration rates by 33%.