Search papers, labs, and topics across Lattice.
This paper introduces a Training-Free Scale-Driven Online Flow Refinement (SOFR) method that enhances the performance of neural video codecs (NVCs) by addressing their limited generalization to complex motion patterns without requiring online fine-tuning. SOFR operates as a plug-and-play module, effectively correcting motion estimation errors by integrating and dynamically fusing motion information from different scales based on warping accuracy. Experimental results demonstrate that SOFR achieves significant bitrate savings of 2.84% and 4.05% in PSNR and MS-SSIM, respectively, for the DCVC-FM framework, while maintaining low computational overhead.
Achieving up to 4.05% bitrate savings in video quality metrics without the need for online fine-tuning could revolutionize how neural video codecs adapt to complex motion.
Although state-of-the-art neural video codecs (NVCs) have achieved remarkable performance, they suffer from limited generalization when encountering complex motion patterns unseen during training. To bridge this domain gap without the expensive cost of online fine-tuning, we propose a Training-Free Scale-Driven Online Flow Refinement (SOFR) method. Serving as a plug-and-play module, SOFR integrates motion information from coarse and fine scales and dynamically fuses them according to warping accuracy, effectively rectifying motion estimation errors with negligible computational overhead. Furthermore, we design a rate-aware strategy that selects different dynamic fusion strategies according to bitrate modes, and employs a reliability check based on warping error to ensure robustness. Extensive experiments on the USTC-TD dataset verify the effectiveness and generalization of SOFR across various NVC frameworks, including DCVC-SDD, DCVC-FM, and EHVC. Notably, it brings an average of 2.84% and 4.05% bitrate savings in terms of PSNR and MS-SSIM, respectively, to DCVC-FM with negligible coding time increase. Our code is available at https://github.com/SunnyMass/SOFR.
