Search papers, labs, and topics across Lattice.
This paper introduces AMNet, a novel multimodal framework for low-light video enhancement (LLVE) that enables modality-agnostic inference, addressing the challenge of information degradation in low-illumination conditions. By employing a Spatial-Spectral Dual-Gated Translator, AMNet learns to generate implicit auxiliary representations from RGB inputs, allowing it to enhance videos even when auxiliary modalities are unavailable. Extensive experiments reveal that AMNet outperforms existing methods, showcasing its robustness in real-world scenarios where multimodal inputs may not be accessible.
AMNet achieves superior low-light video enhancement by generating auxiliary representations on-the-fly, even when critical data modalities are missing.
Low-light video enhancement (LLVE) remains a challenging task due to severe information degradation under low-illumination conditions. Recent multimodal approaches have significantly improved enhancement performance by incorporating auxiliary modalities, such as event streams and infrared images. However, these methods typically assume the availability of these modalities at inference, which is often not feasible in real-world scenarios. To solve this problem, in this work, we propose AMNet, a unified multimodal framework for LLVE, to support flexible modality-agnostic inference, where auxiliary modalities may be unavailable. To address the issue of modality absence, we introduce a Spatial-Spectral Dual-Gated Translator that learns the correspondence between auxiliary modalities and RGB inputs, producing implicit auxiliary representations to support the robust enhancement. Additionally, to fully facilitate the learning of cross-modal correspondence, we conduct large-scale multimodal pretraining based on the RGB-only dataset with synthetic auxiliary modalities. Extensive experiments demonstrate that AMNet could handle arbitrary inference-time modality combinations and exhibits superior performance for LLVE under modality absence conditions. Code and models are available on the project page.
