Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the expressivity of congruence-based layers in deep neural networks specifically designed for classifying symmetric positive-definite matrices. It reveals that imposing semi-orthogonality constraints on the weight matrix significantly restricts the model's capacity, leading to a collapse in expressivity for certain activation functions, which can be traced back to Poincaré's separation theorem. Additionally, the paper evaluates various Riemannian classifiers to determine their compatibility with the feature maps generated by these congruence-like layers, providing insights into optimal classifier selection.
Imposing semi-orthogonality on weight matrices in congruence-based architectures can drastically limit their expressivity, reducing them to one-hidden-layer equivalents for certain activations.
This work studies neural architectures for classifying symmetric positive-definite matrices, focusing on congruence-like layers, in which the input matrix is multiplied on the left and right by a (possibly rectangular) weight matrix $W$ and its transpose. Such layers lie at the core of the celebrated SPDNet and have also been employed independently for dimensionality reduction on positive-definite data. We show that the (semi)-orthogonality constraint commonly imposed on $W$ limits the expressivity of these layers: for certain activation functions, the resulting architecture collapses to a one-hidden-layer equivalent. This lack of expressivity follows from a loss of spectral diversity in congruence-like layers for semi-orthogonal $W$ and is a direct consequence of Poincaré's separation theorem. We then examine the choice of the final classifier, comparing several Riemannian classifiers and discussing their compatibility with the feature maps produced by congruence-like layers.