Search papers, labs, and topics across Lattice.
This study introduces the 3D-anchored Light Probe (3DLP) benchmark, featuring a new high-fidelity HDR dataset that captures real-world lighting changes across 1,000 image pairs of indoor scenes. By evaluating state-of-the-art image editing models on their ability to replicate realistic light probe edits, the research reveals significant variations in model performance, particularly in regions receiving less light. The findings indicate that while top models align closely with real-world lighting physics, there remains substantial room for enhancement, especially in accurately rendering less illuminated areas.
Image editing models may appear visually stunning, but they struggle with accurately reflecting real-world lighting, especially in shadowed regions.
While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these systems possess an intrinsic knowledge of real-world lighting. Existing benchmarks typically evaluate high-level plausibility of perceptual light transport on curated internet imagery, using VLMs or human judgement, or they rely on synthetically generated datasets. In this work, we introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes. The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off. To allow for a granular performance analysis, we annotated specific image regions such as cast shadows or metallic surfaces. With this data, we evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality. The evaluation uses two new scores to compensate for AI-generated photographic effects, such as adjusted white balance. Our results show that the overall performance of models differs considerably, with differences slightly less pronounced for specular highlights. The best image editing models are remarkably consistent with real-world physics, however, they still leave room for improvement. We observe that image regions that receive less light from the light probe are more prone to errors for all models. Furthermore, building on their success in evaluating macroscopic lighting plausibility, we test VLMs on our task but find that they are unsuitable for pixel-level light transport analysis. We will make the benchmark, together with the real-world dataset, publicly available to encourage future research on this topic.
