feature engineering

Existing feature engineering approaches suffer from three fundamental limitations: poor interpretability, weak generalizability, and inflexible strategies—hindering practical deployment across diverse scenarios. To address these challenges, this paper proposes the first large language model (LLM)-driven dynamic adaptive feature generation paradigm. Our method integrates task-aware prompting with semantic modeling of the feature space, enabling real-time, interpretable, and controllable feature generation tailored to both data characteristics and task requirements. It ensures cross-modal and cross-task generality while maintaining full transparency in the feature generation process. Extensive experiments on multiple structured and unstructured data tasks demonstrate that features generated by our approach improve feature quality by 23.6% and boost downstream model performance by an average of 11.4%, significantly outperforming conventional automated feature engineering methods.

This work addresses the limited generalizability of traditional feature engineering, which heavily relies on domain expertise. The authors propose modeling feature engineering as an agent-driven code generation task: an expert agent first produces a structured feature design plan, which a large language model then translates into executable Python code. A dual-channel reinforcement learning framework based on GRPO continuously refines this process by jointly optimizing feature utility and semantic alignment with the original plan. This approach pioneers representing features as executable programs, integrating structured planning, chain-of-thought reasoning, and knowledge transfer to enable cross-domain automated feature engineering. Evaluated on seven public benchmarks, the method significantly outperforms existing AutoFE and LLM-based approaches. In Alibaba Cloud’s GPU resource demand forecasting task, it improves demand fulfillment rate by 16% and reduces resource migration rate by 33%.

In real-world temporal tabular data, the mapping between features and labels continually evolves, inducing temporal distribution shifts that undermine static models’ generalizability and cause adaptive models to overfit transient patterns. To address this, we propose Feature-Aware Temporal Modulation (FATM), the first method to jointly model the co-evolution of features’ objective semantics (statistical distributions) and subjective semantics (task-specific relevance). FATM employs a lightweight time-contextual modulation mechanism to dynamically adjust features’ statistical properties—such as scale and skewness—enabling cross-temporal feature alignment and adaptive mapping. Evaluated on multiple benchmark datasets under distribution shift, FATM significantly improves model robustness and prediction accuracy. It effectively balances generalization and adaptation without architectural complexity or excessive parameter overhead.

In online machine learning, inconsistencies between offline training and online serving feature computation—coupled with inefficient temporal multi-table feature processing—severely hinder real-time performance. To address this, we propose a real-time relational feature computation system tailored for online learning. Our approach introduces: (1) a unified query plan generator to ensure end-to-end feature consistency across the ML lifecycle; (2) a pre-aggregation mechanism with data-adaptive windowing to overcome performance bottlenecks in long-window and multi-table join scenarios; and (3) a time-aware indexing scheme and compact in-memory data format. Built atop a unified SQL engine, the system supports window-level parallelism, time-aware skew mitigation, and streaming memory indexing. Empirical evaluation demonstrates millisecond-scale latency—10× to 100× faster than Flink and Spark—70% lower deployment overhead, and production deployment across 100+ live services. The open-source implementation has garnered 1.6k GitHub stars and 150+ contributors.

This work addresses the disconnect between existing large language model (LLM)-based clinical tabular feature engineering approaches and downstream predictive models, which struggle with class imbalance, heterogeneous features, and stringent interpretability requirements in healthcare data. To bridge this gap, the authors propose MedFeat, a novel framework that, for the first time, integrates model-awareness and interpretability-driven mechanisms into LLM-assisted feature engineering. MedFeat iteratively generates new features that align with the inductive biases of downstream models and are inherently interpretable by fusing feature importance signals with real-time model feedback. Evaluated across multiple real-world clinical prediction tasks, MedFeat significantly outperforms current methods, achieving an average performance improvement exceeding 10%, while maintaining compatibility with diverse model architectures.

Databases continuously evolve through operations such as schema changes, version updates, and data transformations; however, existing approaches typically address these functionalities in isolation, lacking a unified abstraction. This work proposes the first integrated model that unifies continuous schema evolution, version management, and data transformation within a single framework. Built upon general-purpose computational primitives, the model supports operation provenance, conditional update propagation, and change alerts, while employing a declarative mechanism to manage the co-evolution of dependent artifacts—including views and machine learning models. A prototype system implements this framework using an enhanced, parameterized Prolly Tree—a Merkle tree–inspired data structure—to construct a relational-like engine. Experimental evaluation demonstrates that the proposed approach is both feasible and offers tunable performance across diverse evolution scenarios.

This work addresses customs clearance delays in global trade caused by ambiguous product descriptions and frequent updates to Harmonized System (HS) codes. To tackle this challenge, the authors propose a serverless MLOps framework that leverages event-driven pipelines and managed services to enable end-to-end, model-agnostic machine learning lifecycle management. The architecture supports automatic scaling, reproducible training, auditable deployment, and automated A/B testing, ensuring secure and seamless model transitions. By integrating custom text embeddings with models such as Text-CNN, the system achieves 98% accuracy on real-world HS code prediction tasks, meeting stringent service-level agreement (SLA) requirements. This approach significantly reduces long-term operational costs and establishes an efficient, cost-effective, and reproducible deployment paradigm for industrial-scale machine learning systems.

This study addresses the automatic discovery of efficient and interpretable preprocessing and feature engineering methods for structured data, such as time series and tabular datasets. It proposes an Evolutionary Feature Engineering (EFE) framework that, for the first time, employs large language models as evolutionary search operators to automatically generate Python programs conforming to the standard fit/transform interface. The framework iteratively optimizes these programs by leveraging data context, statistical summaries, and validation performance, enabling end-to-end compatibility with existing machine learning pipelines. On time series tasks, the approach reduces MASE, WQL, and MAE errors by over 3% on average (up to 19%). For tabular data, EFE-Tab produces compact feature sets that achieve competitive accuracy with decision tree models while preserving strong interpretability.

Traditional relational database systems offer a monolithic set of features, yet real-world workloads often require only a subset, leading to resource redundancy and suboptimal efficiency. To address this, this work proposes an LLM-driven approach for automatically generating customized, deployable databases from natural language workload descriptions. The method employs Feature-Oriented Domain Analysis (FODA) to decompose databases into modular components and their implementation variants, constructs a dependency graph—DBGraph—augmented with cooperate edges to capture cross-module design constraints, and leverages a multi-agent architecture comprising Main, Architect, Tester, and Refining Agents to orchestrate end-to-end synthesis. Evaluated on the TPC-C benchmark (10 warehouses), the generated system achieves 130 tpmC—outperforming PostgreSQL and MySQL—while comprising only approximately 3% of their codebase and demonstrating zero failures over 60 minutes of continuous operation.

This work addresses the cumbersome and error-prone process of manually extracting samples and labels from relational databases for traditional machine learning modeling. To streamline this workflow, the authors propose PQL, a declarative domain-specific language inspired by SQL that enables users to define diverse predictive tasks—including regression, classification, time-series forecasting, and recommendation—through a single query directly over relational databases, with training labels automatically generated. PQL offers two implementations: one optimized for low-latency, small-scale scenarios and another designed for large-scale data processing. The approach has been validated in real-world applications such as financial fraud detection, product recommendation, and load forecasting, demonstrating its versatility, efficiency, and significant improvements in modeling productivity and scalability.