Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces FF-ProCams, a Feed-Forward 3D Gaussian inverse-rendering framework designed for projector-camera systems, which enhances active scene perception and appearance manipulation. By employing a hybrid Mamba2-Transformer encoder, the method efficiently aggregates geometric and photometric cues from sparse multi-view observations, enabling high-fidelity rendering with a significant reduction in reconstruction time. Experimental results demonstrate that FF-ProCams outperforms traditional optimization-based methods, achieving comparable quality with only 8 input views while reducing test-time reconstruction to just 0.13 seconds.
Achieving high-fidelity projector-aware rendering in a fraction of the time, FF-ProCams outpaces traditional methods by leveraging just 8 input views.
Projector-camera (ProCams) systems achieve active scene perception and controllable appearance manipulation via structured illumination, serving as a core infrastructure for spatial augmented reality, projection mapping, and surface reflectance acquisition. Existing inverse-rendering methods for ProCams deliver high-fidelity results but rely on time-consuming per-scene optimization, while mainstream feed-forward 3D reconstruction models produce baked appearance that cannot adapt to spatially varying projector illumination. To resolve this accuracy-efficiency trade-off, we propose FF-ProCams, a Feed-Forward 3D Gaussian inverse-rendering framework for ProCams. A hybrid Mamba2-Transformer encoder aggregates cross-view geometric and photometric cues from sparse multi-view observations, and lightweight heads predict a relightable Gaussian representation in a single forward pass. We further design a projector-aware differentiable renderer to synthesize camera observations under arbitrary active illumination and ProCams poses. To enable feed-forward training, we construct a large-scale synthetic ProCams dataset covering diverse object geometries and surface materials. Experiments show FF-ProCams achieves high-fidelity projector-aware rendering, generalizes to unseen patterns, and supports novel projector-camera poses. Using only 8 input views, it outperforms optimization-based baselines with 297 views while reducing test-time reconstruction to 0.13 seconds (a three-to-five-order-of-magnitude speedup). The code and data are available at https://github.com/CPREgroup/FF-ProCams/.