Search papers, labs, and topics across Lattice.
LumiTokens introduces a novel framework for 3D relighting that operates directly on latent scene tokens, bypassing the need for explicit 3D representations or physics-based decomposition. By utilizing a Scene Token Editor that employs self-attention to process both scene and light-ray tokens, the system allows for progressive and composable lighting edits in a unified interface. Experimental results show that LumiTokens achieves comparable or superior relighting quality to existing methods while enabling intuitive user interaction with diverse lighting types.
Transforming 3D relighting into a token-space operation allows for intuitive, progressive lighting edits without the overhead of traditional methods.
Existing 3D relighting methods operate through either explicit material decomposition, diffusion-based view-space generation, or a combination of both, requiring full recomputation for each new lighting condition. We observe that recent latent scene representations, which encode multi-view images into a set of compact tokens with no fixed physical semantics, open up a novel design space for relighting. We present LumiTokens, a framework that formulates 3D relighting as a direct transformation on latent scene tokens, without explicit 3D representations, rendering equations, or physics-based decomposition. Our model introduces a Scene Token Editor that processes scene tokens jointly with light-ray tokens through self-attention, producing updated tokens that can be decoded into multi-view-consistent relit images. To support diverse lighting types through a unified interface, all lighting signals, including environment maps, point lights, and area lights, are parameterized as Plucker ray tokens, enabling native 3D user interaction with a representation that carries no explicit spatial structure. Crucially, this design supports progressive relighting: because the editor's output remains in the same latent space as its input, a user can incrementally build up illumination one light source at a time, with each edit composing in token space. Experiments demonstrate that LumiTokens achieves comparable or superior relighting quality to other methods and supports progressive, composable lighting edits. Project page: https://neu-vi.github.io/LumiTokens/
