Search papers, labs, and topics across Lattice.
The paper introduces Any2Any, a unified latent diffusion framework for translating between arbitrary modalities in remote sensing imagery by projecting inputs into a geometrically aligned latent space. This approach addresses the limitations of existing pairwise translation methods, which suffer from quadratic complexity and poor generalization. By leveraging a shared latent representation and lightweight target-specific residual adapters, Any2Any achieves state-of-the-art performance across 14 translation tasks and demonstrates strong zero-shot generalization, enabled by the introduction of RST-1M, a million-scale remote sensing dataset.
Forget training separate models for every remote sensing modality pair: Any2Any learns a single latent space for unified translation, even generalizing to unseen modality combinations.
Multi-modal remote sensing imagery provides complementary observations of the same geographic scene, yet such observations are frequently incomplete in practice. Existing cross-modal translation methods treat each modality pair as an independent task, resulting in quadratic complexity and limited generalization to unseen modality combinations. We formulate Any-to-Any translation as inference over a shared latent representation of the scene, where different modalities correspond to partial observations of the same underlying semantics. Based on this formulation, we propose Any2Any, a unified latent diffusion framework that projects heterogeneous inputs into a geometrically aligned latent space. Such structure performs anchored latent regression with a shared backbone, decoupling modality-specific representation learning from semantic mapping. Moreover, lightweight target-specific residual adapters are used to correct systematic latent mismatches without increasing inference complexity. To support learning under sparse but connected supervision, we introduce RST-1M, the first million-scale remote sensing dataset with paired observations across five sensing modalities, providing supervision anchors for any-to-any translation. Experiments across 14 translation tasks show that Any2Any consistently outperforms pairwise translation methods and exhibits strong zero-shot generalization to unseen modality pairs. Code and models will be available at https://github.com/MiliLab/Any2Any.
