Search papers, labs, and topics across Lattice.
This paper introduces Differential Spectral Damping (DSD), a novel adaptive regularization technique for Least-Squares Twin Support Vector Machines (LSTSVM) that addresses the challenges posed by ill-conditioned matrices due to exponential eigenvalue decay. By tailoring the regularization penalty to the localized eigengap structure, DSD effectively preserves reliable eigenvectors while suppressing corrupted ones, leading to significant improvements in classification accuracy across various datasets. The results show a notable increase in accuracy of up to +10.4 percentage points compared to traditional Tikhonov regularization, demonstrating the method's effectiveness in high-dimensional settings with severe condition numbers.
Tailoring regularization to eigenvector reliability boosts LSTSVM accuracy by up to 10.4 percentage points, challenging conventional damping approaches.
Kernel methods requiring matrix inversion -- particularly Least-Squares Twin Support Vector Machines (LSTSVM) -- suffer from exponential eigenvalue decay in their system matrices, producing severely ill-conditioned problems where standard Tikhonov regularization applies uniform damping regardless of eigenvector reliability. We propose Differential Spectral Damping (DSD), a regularization formula that adapts its penalty to localized eigengap structure: preserving eigenvectors with large spectral gaps (reliable per Davis-Kahan perturbation theory) while aggressively suppressing those with small gaps (directionally corrupted beyond recovery). We motivate DSD through a principled design procedure grounded in the Davis-Kahan $\sin(螛)$ theorem, systematically deriving the requirements for a reliability-aware damping function and selecting the exponential form for its smoothness, differentiability, and natural saturation properties. Through rigorous paired testing with fairly optimized baselines (including gradient-optimized Tikhonov receiving equal optimization opportunity), we demonstrate that DSD improves LSTSVM classification accuracy by +4.8 percentage points on real-world GINA ($d=970$, Cohen's $d = 4.49$, $p < 0.0001$), +10.4 percentage points at $d=200$, and +2.6 percentage points on Madelon ($d=500$) -- all using only principled spectral initialization while Tikhonov receives grid search. For pre-image reconstruction on manifold data, DSD ties Tikhonov at high perturbation noise ($p=0.99$) but slightly underperforms at lower noise levels; both reduce naive inversion error by $66\times$. We characterize the precise operating regime ($d \geq 100$, condition number $> 10^3$) and document where simpler methods suffice, providing practitioners with clear deployment guidance.
