Search papers, labs, and topics across Lattice.
This paper investigates the linear independence of polynomial compositions and their implications for the identifiability of deep neural networks, conjecturing that composing a fixed number of distinct nonconstant polynomials with a sufficiently high-degree polynomial results in linear independence. The authors establish several cases supporting this conjecture, including proofs for two polynomials and for multiple polynomials with bounded degrees. Notably, the findings provide a comprehensive characterization of parameter symmetries in fully connected neural networks with polynomial activations, resolving the identifiability of shallow polynomial networks.
Composing distinct polynomials can lead to linear independence, fundamentally reshaping our understanding of neural network identifiability.
Motivated by theoretical problems in deep learning, we conjecture that post-composing a fixed number of pairwise distinct nonconstant polynomials with a generic polynomial of sufficiently large degree yields linearly independent polynomials. This generalizes Newman--Slater's theorem on powers of polynomials. We establish several cases of this conjecture and its origin-passing variant: We prove the result for two polynomials, and for an arbitrary number of polynomials when their degrees are bounded. Furthermore, we show how the conjecture implies a complete understanding of the identifiability (i.e., parameter symmetries) of deep fully connected neural network architectures with generic polynomial activation functions. In particular, for network architectures with layer-specific activations of increasing degree, our established versions of the conjecture fully characterize the set of parameters yielding the same end-to-end network function. As a special case, we fully resolve the identifiability of shallow polynomial networks.
