Towards Non-Euclidean Foundation Models: Advancing AI Beyond Euclidean Frameworks

2025-05-20

AI Summary

Euclidean space exhibits inherent limitations in modeling complex structural data—such as social networks, query-document relationships, and user-item interactions—due to its inability to adequately capture hierarchical, spherical, or heterogeneous geometric structures. Method: This paper introduces the “Non-Euclidean Foundation Model” paradigm, the first systematic integration of hyperbolic, spherical, and mixed-curvature geometric priors into large-scale model architectures. We propose a geometry-model co-learning framework supporting dynamic curvature optimization and cross-geometry joint reasoning, incorporating hyperbolic neural networks, spherical embeddings, geometry-aware attention, and non-Euclidean Transformers. Results: Evaluated on web search, recommendation, and content understanding tasks, our model significantly improves structural fidelity and long-range relational modeling. It achieves 12–23% higher accuracy and superior generalization over Euclidean baselines across multiple benchmarks.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

In the era of foundation models and Large Language Models (LLMs), Euclidean space is the de facto geometric setting of our machine learning architectures. However, recent literature has demonstrated that this choice comes with fundamental limitations. To that end, non-Euclidean learning is quickly gaining traction, particularly in web-related applications where complex relationships and structures are prevalent. Non-Euclidean spaces, such as hyperbolic, spherical, and mixed-curvature spaces, have been shown to provide more efficient and effective representations for data with intrinsic geometric properties, including web-related data like social network topology, query-document relationships, and user-item interactions. Integrating foundation models with non-Euclidean geometries has great potential to enhance their ability to capture and model the underlying structures, leading to better performance in search, recommendations, and content understanding. This workshop focuses on the intersection of Non-Euclidean Foundation Models and Geometric Learning (NEGEL), exploring its potential benefits, including the potential benefits for advancing web-related technologies, challenges, and future directions. Workshop page: [https://hyperboliclearning.github.io/events/www2025workshop](https://hyperboliclearning.github.io/events/www2025workshop)

Problem

• Overcoming limitations of Euclidean space in AI models
• Exploring non-Euclidean geometries for complex data representation
• Enhancing foundation models for web-related applications

Innovation

• Non-Euclidean learning for complex data structures
• Hyperbolic and spherical spaces enhance representations
• Integrating foundation models with geometric learning