Search papers, labs, and topics across Lattice.
This paper introduces Flow-Map Distillation on Relation Manifolds (FoRM), which reformulates knowledge transfer in image restoration as a continuous flow mapping problem, allowing for dynamic supervision of relation states between teacher and student networks. By employing a flow map operator that predicts relation states over time and enforcing consistency through a semigroup constraint, FoRM significantly reduces phantom-state errors and enhances the overall training process. Experimental results across five image restoration tasks reveal that FoRM outperforms state-of-the-art methods, achieving approximately 50% reduction in training variance while improving restoration quality.
Flow-Map Distillation achieves superior image restoration by transforming static knowledge transfer into a dynamic flow mapping process, cutting training variance in half.
Knowledge distillation for image restoration typically aligns intermediate features or relation matrices between teacher and student networks as static targets, ignoring the dynamic structure of the knowledge transfer process. In this paper, we propose Flow-Map Distillation on Relation Manifolds (FoRM), which reformulates relation-based knowledge transfer as a continuous flow mapping problem on the relation manifold. Rather than regressing a constant velocity field between student and teacher relation states, FoRM learns a flow map operator $\mathcal{F}_\theta(\mathbf{z}, t, s)$ that directly predicts the relation state at any target time $s$ given the current state at time $t$, enabling richer trajectory-level supervision. To ensure global self-consistency of the learned flow map, we introduce a safe semigroup consistency constraint that enforces compositional agreement using ground-truth bridge states, eliminating phantom-state error accumulation. An endpoint anchoring loss further prevents the operator from drifting away from the teacher target. Extensive experiments on five image restoration tasks, including super-resolution, deraining, denoising, deblurring, and low-light enhancement, demonstrate consistent gains over state-of-the-art distillation baselines across multiple backbone architectures, reducing training variance by approximately 50\% compared to naive flow matching distillation while achieving superior restoration quality.
