Improving Set Function Approximation with Quasi-Arithmetic Neural Networks

2026-02-04

AI Summary

This work proposes Quasi-Arithmetic Neural Networks (QUANNs), a novel approach that overcomes the limitations of traditional set models, which rely on fixed pooling operations such as summation or max-pooling and thereby constrain the expressiveness and transferability of learned embeddings. QUANNs uniquely integrate invertible neural networks with generalized measures of central tendency to construct a learnable, theoretically grounded Neuralized Kolmogorov Mean (NKM) aggregator capable of efficiently approximating arbitrary set functions. Empirical evaluations demonstrate that the proposed method outperforms current state-of-the-art models across multiple benchmark tasks. Moreover, the learned embeddings exhibit higher structural coherence and display strong transfer performance even in non-set-based downstream tasks, highlighting the model’s versatility and representational power.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such as DeepSets and PointNet rely on fixed, non-learnable pooling operations (e.g., sum or max) -- a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed as more expressive alternatives. In this work, we advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) -- a novel, trainable framework for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as a learnable aggregation function. We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions and, thanks to their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.

Problem

• set function approximation
• learnable aggregation
• model expressivity
• embedding transferability
• unordered data

Innovation

• Neuralized Kolmogorov Mean
• quasi-arithmetic neural networks
• learnable aggregation
• set function approximation
• invertible neural networks