Arnav Chavan

Restricting LLM fine-tuning to the bottom singular subspace nearly eliminates catastrophic forgetting, even when protecting 99% of weights, while maintaining strong target-task performance.

Achieving a 50% reduction in active parameters while maintaining 90% of the original model's performance could redefine efficiency benchmarks for LLMs.