Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer

2026-05-08

AI Summary

This work addresses the limited interpretability of embeddings in health foundation models and the challenge of post-training cross-modal knowledge transfer. The authors propose a training-free post-processing framework that decomposes frozen embeddings to extract interpretable symbolic directions highly correlated with health states. These symbolic representations are then leveraged to align embedding spaces across distinct modalities—such as photoplethysmography (PPG) and accelerometer data—by identifying a shared low-dimensional subspace of physiological information. Requiring only minimal paired data, the method achieves efficient cross-modal alignment while preserving over 95% of within-domain performance and substantially enhancing cross-domain transfer efficacy.

Abstract

Health foundation models (FMs) learn useful representations from wearable sensors, but interpreting what they encode and transferring that knowledge across modalities after training remains difficult. We present a post-training framework that decomposes frozen embeddings into interpretable directions, referred to as symbols, and use these symbols to align the embedding spaces without retraining. We evaluate the framework on three FMs for photoplethysmography (PPG) and accelerometer data, independently pretrained on ~20M minutes of unlabeled data from ~172K participants, and analyzed on a held-out cohort of 30K subjects. We find that extracted symbols associate selectively with health conditions and physiological attributes, and these associations are partially shared across modalities and architectures. Cross-modal transfer via symbols retains more than 95% of in-domain performance, is nearly symmetric across domain directions, and saturates with limited paired data, together indicating that alignment recovers a shared low-dimensional subspace rich in physiological information. Overall, these results suggest that health FM embeddings contain an interpretable symbolic organization that is shared across modalities and supports cross-domain transfer without joint training.

Problem

• health foundation models
• interpretability
• cross-modal transfer
• symbolic structure
• embedding alignment

Innovation

• symbolic representation
• cross-modal alignment
• foundation models
• post-training interpretability
• physiological embedding