Search papers, labs, and topics across Lattice.
This paper introduces RaysUp, an ultra-lightweight and task-agnostic framework for feature upsampling that reconstructs high-resolution feature maps from low-resolution outputs of Vision Foundation Models (VFMs). By leveraging a geometry-aware ray representation and innovative components such as a Spatially Decoupled Guidance Encoder and Any-Resolution Cross-Attention, RaysUp achieves superior semantic fidelity without the need for VFM-specific retraining. Extensive evaluations show that RaysUp outperforms existing methods with only 16% of the parameters of AnyUp and provides approximately 7x faster inference, marking a significant advancement in the accuracy-efficiency trade-off for dense prediction tasks.
RaysUp achieves state-of-the-art feature upsampling performance with just 16% of the parameters of existing methods, revolutionizing efficiency in dense prediction tasks.
Pre-trained Vision Foundation Models (VFMs) have become central to modern computer vision due to their powerful semantic representations and strong generalization ability. However, their patchified or pooled outputs are inherently low-resolution, limiting their effectiveness in tasks requiring fine-grained, pixel-level reasoning. Existing feature upsampling approaches either degrade semantic fidelity or rely on VFM-specific retraining and heavy architectures, hindering efficiency and scalability. To address these challenges, we propose RaysUp, an ultra-lightweight, task-agnostic, and VFM-agnostic feature upsampling framework that reconstructs high-resolution feature maps at arbitrary resolutions. Unlike conventional 2D interpolation or attention-based schemes, RaysUp lifts feature reconstruction into a geometry-aware ray domain. Specifically, we introduce a Spatially Decoupled Guidance Encoder for direction-aware guidance encoding, an Any-Resolution Cross-Attention mechanism for resolution-flexible reconstruction, and a novel Ray Positional Encoding (RayPE) that injects implicit 3D geometric priors via 6D Plucker ray coordinates. Finally, a Geometry-Aware Neighborhood Attention module further ensures content-adaptive bilateral aggregation while preserving geometric consistency. Extensive experiments across diverse dense prediction tasks demonstrate that RaysUp achieves state-of-the-art performance while using only 16% of the parameters of AnyUp and delivering approximately 7x faster inference. These results highlight a substantially improved accuracy-efficiency trade-off and establish RaysUp as a practical and scalable solution for universal feature upsampling. Code is available at https://github.com/MAP-RaysUp/RaysUp.
