Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper extends the entropy formula originally proposed by Menon and Yu for real Deep Linear Networks (DLNs) to encompass complex and quaternionic DLNs, thereby providing a unified framework for analyzing these networks across different mathematical domains. This extension is significant as it allows for a deeper understanding of the information dynamics within DLNs, which can influence model performance and interpretability. The key result is the derivation of a comprehensive entropy formula that applies to real, complex, and quaternionic DLNs, facilitating further research in these areas.
A unified entropy formula for deep linear networks reveals critical insights into information flow across real, complex, and quaternionic domains.
We extend the entropy formula of Menon and Yu for the real Deep Linear Network (DLN) to its complex and quaternionic analogues, obtaining a unified formula for DLNs over $\mathbb{R}$, $\mathbb{C}$, and $\mathbb{H}$.