Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networks

2026-08-27

AI Summary

研究通过证明多项式组合的线性独立性来解决深度神经网络的可识别性问题，特别针对具有特定激活函数的网络架构。

Abstract

Motivated by theoretical problems in deep learning, we conjecture that post-composing a fixed number of pairwise distinct nonconstant polynomials with a generic polynomial of sufficiently large degree yields linearly independent polynomials. This generalizes Newman--Slater's theorem on powers of polynomials. We establish several cases of this conjecture and its origin-passing variant: We prove the result for two polynomials, and for an arbitrary number of polynomials when their degrees are bounded. Furthermore, we show how the conjecture implies a complete understanding of the identifiability (i.e., parameter symmetries) of deep fully connected neural network architectures with generic polynomial activation functions. In particular, for network architectures with layer-specific activations of increasing degree, our established versions of the conjecture fully characterize the set of parameters yielding the same end-to-end network function. As a special case, we fully resolve the identifiability of shallow polynomial networks.

Problem

• Linear Independence
• Polynomial Compositions
• Identifiability
• Deep Neural Networks
• Activation Functions

Innovation

• linear independence
• polynomial compositions
• identifiability
• deep neural networks
• activation functions