Multi-context principal component analysis

2026-01-21

AI Summary

This work proposes Multi-Context Principal Component Analysis (MCPCA), a systematic extension of traditional PCA to settings involving multiple contexts—such as distinct diseases, cell types, or textual corpora—where conventional PCA fails to capture latent factors shared only across subsets of contexts. By introducing a shared factorization framework and a tailored optimization algorithm, MCPCA precisely uncovers common variation structures among arbitrary context subsets. Applied to gene expression data, the method identifies a tumor-cell-specific axis of variation linked to lung cancer progression; in language model embeddings, it reveals evolutionary trajectories in scientific and fictional discourse on human nature spanning decades. These findings demonstrate MCPCA’s capacity to overcome the limitations of standard PCA, which is constrained to either single-context or naively pooled multi-context analyses.

Technology Category

• Machine Learning: Learning with Manifolds
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Data Compression

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Principal component analysis (PCA) is a tool to capture factors that explain variation in data. Across domains, data are now collected across multiple contexts (for example, individuals with different diseases, cells of different types, or words across texts). While the factors explaining variation in data are undoubtedly shared across subsets of contexts, no tools currently exist to systematically recover such factors. We develop multi-context principal component analysis (MCPCA), a theoretical and algorithmic framework that decomposes data into factors shared across subsets of contexts. Applied to gene expression, MCPCA reveals axes of variation shared across subsets of cancer types and an axis whose variability in tumor cells, but not mean, is associated with lung cancer progression. Applied to contextualized word embeddings from language models, MCPCA maps stages of a debate on human nature, revealing a discussion between science and fiction over decades. These axes are not found by combining data across contexts or by restricting to individual contexts. MCPCA is a principled generalization of PCA to address the challenge of understanding factors underlying data across contexts.

Problem

• multi-context
• principal component analysis
• shared factors
• data variation
• cross-context

Innovation

• multi-context PCA
• shared latent factors
• cross-context data analysis
• contextualized embeddings
• heterogeneous data decomposition