AURORA: Contextual Orthogonalization for Geometric Representation Learning in Healthcare Foundation Models

2026-05-17

AI Summary

This work addresses the semantic entanglement commonly observed in latent representations of medical foundation models, where factors such as physiological severity, intervention intensity, and institutional workflows are conflated, leading to opaque semantics and context instability. To resolve this, the authors propose AURORA, a novel framework that explicitly embeds contextual semantic structure into the geometry of the latent space. AURORA achieves semantic disentanglement through orthogonal subspace decomposition and optimizes within each subspace for relational consistency and uncertainty-aware objectives. Departing from conventional paradigms that rely solely on predictive compression, AURORA significantly outperforms baseline approaches—including reconstruction, contrastive learning, and self-distillation—in both clinical prediction and retrieval tasks. The method markedly enhances representation disentanglement, neighborhood purity, and robustness under cross-institutional distribution shifts.

Abstract

Recent healthcare foundation models have achieved strong predictive performance through large scale self supervised learning, yet their latent representations frequently entangle physiologic severity, intervention intensity, observational structure, and institutional workflow into shared embedding directions. While effective for downstream prediction, such representations remain semantically opaque and unstable under contextual shift. We introduce AURORA, Adaptive Uncertainty aware Representations through Orthogonalized Relational Alignment, a new framework for healthcare representation learning based on contextual latent geometry. Rather than optimizing a single unified embedding manifold, AURORA decomposes representations into orthogonal semantic subspaces corresponding to distinct contextual factors and learns relational consistency objectives within each subspace. This induces latent spaces that are both semantically disentangled and geometrically interpretable. Across multiple clinical prediction and retrieval tasks, AURORA consistently outperforms reconstruction, contrastive, and self distillation baselines while substantially improving contextual disentanglement, neighborhood purity, and robustness under institutional distribution shift. Our results suggest that latent geometry itself constitutes an important axis of healthcare foundation model design and that explicitly structuring representation space according to contextual semantics provides a complementary direction beyond conventional predictive compression objectives.

Problem

• healthcare foundation models
• representation disentanglement
• contextual shift
• latent geometry
• semantic opacity

Innovation

• orthogonalization
• contextual disentanglement
• geometric representation learning
• healthcare foundation models
• relational alignment