Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search

2026-09-09

AI Summary

该研究针对特征转换中的层次关系忽视、顺序敏感性和非凸空间搜索问题，提出了一种结合自注意力机制和策略引导强化学习的框架来优化表格式数据的预测性能。

Abstract

Feature transformation improves predictive performance on tabular data by constructing informative abstractions from raw features. Recent generative approaches encode transformation knowledge into continuous embedding spaces for efficient exploration of candidate strategies, but face three key limitations: (1) overlooking hierarchical relationships between low-level features, operations, and high-level abstractions; (2) enforcing order-sensitive embeddings on inherently permutation-invariant transformation sequences, thereby introducing systematic bias; and (3) relying on gradient-based search, which is ill-suited to non-convex transformation spaces. We propose a framework with two complementary components. First, a permutation-invariant hierarchical module captures interactions across features, operations, and abstraction levels, with a self-attention pooling mechanism that maps semantically equivalent structures to consistent embeddings aligned with downstream performance. Second, a policy-guided multi-objective reinforcement learning strategy initializes the search from empirically strong seeds and jointly optimizes predictive accuracy and transformation efficiency. Extensive experiments on diverse tabular benchmarks demonstrate the effectiveness and robustness of our framework against strong baselines. Our code and data are publicly available at: https://github.com/RayLiu1103/PHER.

Problem

• Feature Transformation
• Hierarchical Relationships
• Permutation-Invariant
• Gradient-Based Search

Innovation

• permutation-invariant
• hierarchical module
• self-attention pooling
• policy-guided reinforcement learning
• multi-objective optimization