Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
LiteReality-Agent reconstructs realistic, articulated, and simulation-ready 3D indoor scenes from RGB-D scans by framing the problem as a coding task for an agent. The agent iteratively edits a Python script, Room.py, using specialized tools within an observe-edit-verify harness to manage evidence gathering, measurement, layout optimization, and quality control. This approach yields reconstructions superior in geometric accuracy, visual realism, and simulation compatibility compared to recent frontier models, enabling robust real-to-sim systems.
An agent can generate high-fidelity, simulation-ready 3D indoor scene reconstructions by iteratively editing Python code, outperforming frontier models.
We present LiteReality-Agent, an agentic system for reconstructing real indoor environments as realistic, articulated, and simulation-ready 3D scenes from RGB-D scans. At its core, LiteReality-Agent formulates 3D reconstruction as a coding problem, in which a coding agent gathers evidence using specialised tools and iteratively edits a Python script, Room.py, which can be executed to produce a 3D digital twin of the room. With this formulation, we develop a robust observe-edit-verify harness that supports evidence gathering, measurement, verification, layout optimisation, simulation readiness, and quality control throughout the reconstruction process. LiteReality-Agent produces high-quality reconstructions suitable for simulation and downstream embodied AI tasks. Furthermore, as agent capabilities continue to improve rapidly, the system introduced by LiteReality-Agent remains a strong orchestration framework for future agents: it equips them with specialised tools, structured workflows, and robust verification mechanisms that substantially improve reconstruction quality and reliability. We demonstrate that LiteReality-Agent produces reconstructions that are more geometrically accurate, visually realistic, and simulation-compatible than those generated by recent frontier models, such as Astra and Fable. We therefore view LiteReality-Agent as a practical and important building block for robust real-to-sim systems. Both the source code and the data-capture application are publicly available. Code:https://github.com/LiteReality/LiteReality-Agent/