Towards Universal Neural Inference

2025-08-12

AI Summary

Real-world heterogeneous structured data lack unified schemas, exhibit semantic inconsistencies, and possess arbitrary feature orderings, severely hindering cross-dataset generalization. To address this, we propose ASPIRE—the first general-purpose neural reasoning model for open-domain heterogeneous tabular data. Its core innovation lies in a permutation-invariant set-based Transformer architecture, augmented by semantic anchoring and multi-source semantic grounding (natural language descriptions, metadata, and contextual examples) to enable cross-table semantic alignment and dynamic reasoning. ASPIRE is the first to support zero-shot task transfer without fine-tuning and jointly optimizes prediction accuracy with budget-constrained active feature acquisition at inference time. Experiments across multiple benchmarks demonstrate that ASPIRE significantly outperforms existing methods, robustly generalizes to unseen datasets, and effectively performs both semantic reasoning and target prediction.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Real-world data often appears in diverse, disjoint forms -- with varying schemas, inconsistent semantics, and no fixed feature ordering -- making it challenging to build general-purpose models that can leverage information across datasets. We introduce ASPIRE, Arbitrary Set-based Permutation-Invariant Reasoning Engine, a Universal Neural Inference model for semantic reasoning and prediction over heterogeneous structured data. ASPIRE combines a permutation-invariant, set-based Transformer with a semantic grounding module that incorporates natural language descriptions, dataset metadata, and in-context examples to learn cross-dataset feature dependencies. This architecture allows ASPIRE to ingest arbitrary sets of feature--value pairs and support examples, align semantics across disjoint tables, and make predictions for any specified target. Once trained, ASPIRE generalizes to new inference tasks without additional tuning. In addition to delivering strong results across diverse benchmarks, ASPIRE naturally supports cost-aware active feature acquisition in an open-world setting, selecting informative features under test-time budget constraints for an arbitrary unseen dataset. These capabilities position ASPIRE as a step toward truly universal, semantics-aware inference over structured data.

Problem

• Handling diverse disjoint data with varying schemas and semantics
• Building general-purpose models for cross-dataset feature dependencies
• Supporting cost-aware active feature acquisition under budget constraints

Innovation

• Set-based Transformer for permutation-invariant reasoning
• Semantic grounding module with natural language descriptions
• Cost-aware active feature acquisition under budget constraints