Search papers, labs, and topics across Lattice.
RecycleLoRA enhances domain generalization in semantic segmentation by using Rank-Revealing QR Decomposition (RRQR) to exploit subspace structures within Vision Foundation Models (VFMs). It introduces a dual-LoRA adapter design: one adapter learns diverse features from minor subspace directions, while the other refines major directions. This approach achieves state-of-the-art performance on domain generalization tasks without complex architectures or increased inference latency, demonstrating the effectiveness of RRQR-based initialization for subspace exploitation.
RecycleLoRA reveals that strategically targeting minor subspace directions in VFMs with LoRA adapters can unlock surprisingly robust domain generalization in semantic segmentation.
Domain Generalized Semantic Segmentation (DGSS) aims to maintain robust performance across unseen target domains. Vision Foundation Models (VFMs) offer rich multi-domain knowledge that can enhance generalization. However, strategies for actively exploiting the rich subspace structures within VFMs remain under-explored, with many existing methods focusing primarily on preserving pre-trained knowledge. Furthermore, their LoRA components often suffer from limited representational diversity and inefficient parameter utilization. We propose RecycleLoRA, which addresses both challenges by employing Rank-Revealing QR Decomposition (RRQR) to systematically exploit VFM's subspace structures and enhance LoRA's representational richness. Our main adapter leverages minor subspace directions identified by RRQR to learn diverse and independent features, achieving competitive performance even when used alone. We further introduce a sub adapter that carefully refines major directions with minimal adjustments, providing complementary improvements to the main adapter's strong baseline performance. This design enables the dual adapters to learn distinct representations without requiring additional regularization losses. Our systematic exploitation of pre-trained subspace structures through RRQR-based initialization leads to superior domain generalization performance. RecycleLoRA achieves state-of-the-art performance on both synthetic-to-real generalization and real-to-real generalization tasks without complex architectures or additional inference latency.
