Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
We extend Kernel Singular Value Decomposition (KSVD) to handle multiple asymmetric kernelized data sources via eKSVD, performing joint nonlinear feature learning with pair-wise couplings. The primal formulation jointly learns projections to maximize information capture, while the dual optimization yields a generalization of the shifted eigenvalue problem. A complementary covariance-based framework with neural network feature mappings is also derived, demonstrating superior performance over Mercer kernel methods for multi-source data.
Jointly learning nonlinear features across multiple asymmetric data sources unlocks performance gains beyond traditional kernel methods.
Kernel Singular Value Decomposition (KSVD) learns a pair of singular vectors w.r.t. an asymmetric kernel matrix, which can be induced by two data sources, e.g., the queries and keys in self-attention or the rows and columns of a given matrix. In this work, we extend KSVD to multiple data sources, namely eKSVD, which conducts joint nonlinear feature learning upon asymmetric kernels. In the primal formulation, the projections associated with each data source are jointly learned to capture maximal information, while incorporating pair-wise couplings. With the Lagrangian and its Karush-Kuhn-Tucker (KKT) conditions, the optimization in the dual leads to a generalization of the shifted eigenvalue problem in Lanczos decomposition theorem of KSVD. Further, a covariance-based framework is derived together with using neural networks (NNs) for explicit feature mappings, complementary to the kernel-based interpretation and optimization. Numerical experiments verify the effectiveness of our eKSVD compared to methods based on Mercer kernels for tackling multiple data sources, and our innovation of deploying NNs demonstrates great flexibility for kernel methods.