🤖 AI Summary
Standard supervised learning often disregards the geometric structure of the feature space, leading to entangled representations that are highly sensitive to label noise and input perturbations. This work proposes Class-Conditional Activation Regularization (CCAR), which introduces a block-diagonal soft inductive bias to constrain the feature energy of each class within orthogonal subspaces, thereby explicitly constructing disentangled representations endowed with an intrinsic geometric skeleton. Theoretically, this study establishes, for the first time, a formal connection between geometric disentanglement and algorithmic stability, demonstrating that robustness arises from a well-structured feature space—achieved by maximizing the Fisher discriminant ratio. Extensive experiments show that CCAR significantly outperforms existing methods across multiple benchmarks involving label noise and input corruptions, confirming its effectiveness and enhanced robustness.
📝 Abstract
Standard supervised learning optimizes for predictive accuracy but remains agnostic to the internal geometry of learned features, often yielding representations that are entangled and brittle. We propose Class-Conditional Activation Regularization (CCAR) to explicitly engineer the feature space, imposing a block-diagonal structure via a soft inductive bias. By shaping the latent representation to confine class energy to orthogonal subspaces, we create an intrinsic geometric scaffold that naturally filters noise and adversarial perturbations. We provide theoretical analysis linking this structural constraint to the maximization of the Fisher Discriminant Ratio, establishing a formal connection between geometric disentanglement and algorithmic stability. Empirically, this approach demonstrates that robustness is an emergent property of a well-engineered feature space, significantly outperforming baselines on label noise and input corruption benchmarks.