Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

2026-05-08

AI Summary

Deep neural networks often fail in out-of-distribution (OOD) generalization due to their inability to learn transferable representations from in-distribution data alone. This work demonstrates that OOD extrapolation under a single training distribution is fundamentally unidentifiable and argues that “structural commitments”—joint constraints on feature mappings, label mappings, and model classes—are essential for effective OOD generalization. Theoretical analysis and experiments reveal that modifying only the feature representation can lead to performance differences of up to 520× in OOD settings. The proposed framework is validated across diverse architectures—including Transformers, Mamba, and S4D—and tasks spanning Fourier coordinates, periodic extrapolation, and scientific domains such as chemical reaction prediction, exoplanet detection, and cross-species DNA analysis, showing that while appropriate features are necessary, they must be matched with sufficient model expressivity and adequately covered representation spaces.

Abstract

Successful deep neural networks discover salient features of data. We show when and why they fail to learn out-of-distribution (OOD)-relevant representations from an in-distribution (ID) training window. This requires decoupling feature learning from data-generating-process (DGP) identifiability. From a single training window, OOD extrapolation is non-identifiable: infinitely many DGPs are $\varepsilon$-observationally equivalent on the training data but diverge arbitrarily outside it, and no in-distribution criterion alone reliably breaks the tie. A structural commitment, the feature map, label map, and model class $(\varphi, ψ, \mathcal{M})$, dictates the assumed DGP and governs OOD generalization while leaving ID performance essentially unchanged. When architecture, pretraining, augmentation, input formats, or domain knowledge implicitly inject the missing commitment, the model succeeds. When it cannot infer OOD-relevant structure from ID evidence, it fails. Changing only the representation can make the same architecture, at the same in-distribution loss, differ by ${\sim}520\times$ out of distribution. When the commitment is correct and identifiable, OOD error vanishes. For example, Fourier coordinates turn periodic extrapolation into interpolation on $\mathbb{S}^1$. The same mechanism predicts outcomes in three natural-science settings (mass-action chemistry; Kepler's-third-law exoplanet prediction, $n=2{,}362$; and cross-species coding-DNA detection) and in a 264-run positional-encoding study across Transformer, Mamba, and S4D. Finally, a controlled study shows: correct features are necessary but not sufficient. The model class must express the target, and the transformed training data must cover the relevant representation space.

Problem

• out-of-distribution generalization
• identifiability
• feature engineering
• extrapolation
• data-generating process

Innovation

• out-of-distribution generalization
• feature engineering
• identifiability
• data-generating process
• representation learning