Dimension Agnostic Neural Processes

2025-02-28

AI Summary

Existing neural processes (NPs) struggle to handle variable input dimensions and multi-scale features, limiting their generalization capability in meta-learning-based uncertainty regression. To address this, we propose Dimension-Independent Neural Processes (DI-NP). First, we design a Dimension Aggregation Block (DAB) that enables adaptive normalization over input spaces of arbitrary dimensionality. Second, we integrate a Transformer architecture with implicit latent variable encoding to enhance cross-task feature representation generality. Third, we embed an explicit uncertainty modeling mechanism to ensure prediction reliability. Evaluated on diverse synthetic and real-world regression benchmarks, DI-NP consistently outperforms state-of-the-art NP variants—demonstrating superior generalization, robustness to distributional shifts, and cross-dataset transferability. Our approach establishes a new paradigm for meta-regression under high-dimensional, heterogeneous inputs.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Meta-learning aims to train models that can generalize to new tasks with limited labeled data by extracting shared features across diverse task datasets. Additionally, it accounts for prediction uncertainty during both training and evaluation, a concept known as uncertainty-aware meta-learning. Neural Process(NP) is a well-known uncertainty-aware meta-learning method that constructs implicit stochastic processes using parametric neural networks, enabling rapid adaptation to new tasks. However, existing NP methods face challenges in accommodating diverse input dimensions and learned features, limiting their broad applicability across regression tasks. To address these limitations and advance the utility of NP models as general regressors, we introduce Dimension Agnostic Neural Processes(DANP). DANP incorporates Dimension Aggregator Block(DAB) to transform input features into a fixed-dimensional space, enhancing the model's ability to handle diverse datasets. Furthermore, leveraging the Transformer architecture and latent encoding layers, DANP learns a wider range of features that are generalizable across various tasks. Through comprehensive experimentation on various synthetic and practical regression tasks, we empirically show that DANP outperforms previous NP variations, showcasing its effectiveness in overcoming the limitations of traditional NP models and its potential for broader applicability in diverse regression scenarios.

Problem

• Handles diverse input dimensions in regression tasks
• Improves feature generalization across various tasks
• Enhances uncertainty-aware meta-learning with fixed-dimensional space

Innovation

• Dimension Agnostic Neural Processes (DANP) introduced
• Dimension Aggregator Block (DAB) transforms input features
• Transformer architecture enhances feature generalization