Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation

2026-06-16

AI Summary

This work addresses the challenge of simultaneously ensuring identifiability and extrapolation capability in conditional latent variable models—specifically, how variations in observed attributes shape the latent structure and generalize to unseen attribute values. The authors propose Concept Modulation Models (CMMs), which formalize a generative pathway from attributes through modulators to concepts and ultimately to observations. By introducing an attribute potential function, they unify the characterization of conditional identifiability and extrapolation behavior. The study extends transfer-based identifiability theory to the conditional setting for the first time, establishing a general algebraic criterion that jointly governs identifiability and extrapolation. This framework reproduces and unifies foundational theoretical results from nonlinear independent component analysis and causal representation learning.

Abstract

Reliable generalization in conditional latent variable models requires understanding both identifiability and extrapolation: how observed variation across attributes determines latent structure, and how that structure determines distributions at unseen attributes. However, existing identifiability and extrapolation guarantees are largely model-specific, with separate analyses in nonlinear ICA, causal representation learning, perturbation modeling, and related conditional latent variable models. We introduce concept modulation models (CMMs), an attribute-indexed class of conditional generative models with structure $A\to Λ\to C\to X$, where attributes select modulators, modulators induce latent concept laws, and concepts generate observed features. CMMs lift transition-based identifiability to conditional settings by showing that feature agreement on observed attributes induces a latent concept transition constrained by the CMM class. We express these constraints through attribute potentials, log-density ratios between attribute-conditioned concept laws, separating the generic lifting step from model-specific rigidity arguments. The same potentials control extrapolation: agreement at unseen attributes holds exactly when the transported attribute-potential identities extend to those attributes. This yields algebraic extrapolation criteria, identifies the common potential-based proof objects behind several existing identifiability and extrapolation results, and, when combined with the model-specific rigidity arguments in those works, recovers their stated conclusions.

Problem

• identifiability
• extrapolation
• conditional latent variable models
• concept modulation
• attribute generalization

Innovation

• concept modulation models
• identifiability
• extrapolation
• attribute potentials
• conditional latent variable models