Search papers, labs, and topics across Lattice.
This paper introduces CoCo, a novel loss function designed to enhance the learning of normalized and structured representations in neural networks. By promoting intra-class collapse and inter-class contrast, CoCo enables the approximation of geometrically optimal embeddings with significant angular separation between classes. Experimental results on the OpenML-CC18 benchmark reveal that CoCo not only competes with state-of-the-art methods but also accelerates convergence and improves class clustering, demonstrating its effectiveness in learning discriminative representations.
CoCo loss achieves faster convergence and tighter class clustering, outperforming traditional methods in learning optimal embeddings.
In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations. The proposed loss encourages intra-class collapse and inter-class contrast while preserving sufficient flexibility for neural networks to approximate geometrically optimal embeddings with large angular separation between classes. We provide a theoretical analysis positioning CoCo with respect to related objectives such as dot regression and cross-entropy, showing that the new proposed loss benefits from closer initialization to the optimal configuration, more informative gradients, and stronger incentives for class-wise representation collapse. Extensive experiments on diverse tabular datasets from the OpenML-CC18 benchmark show that CoCo achieves competitive performance with state-of-the-art methods, including kernel SVM, Random Forest, dot regression, and cross-entropy-based neural networks. In addition, both theoretical arguments and empirical analyses demonstrate that the proposal promotes tighter class clustering and faster convergence. These results highlight CoCo loss as an effective objective for learning discriminative representations while maintaining competitive predictive performance.
