Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to urban layout extraction from remote-sensing images using a multimodal large language model (MLLM) that generates executable city layouts. By first producing a semantic layout prior and then employing three ordered MLLM passes, the method effectively recovers urban elements such as roads, land-cover regions, and buildings, culminating in a city graph and a restricted layout program. The framework achieves a mean intersection-over-union of 41.1% and a global intersection-over-union of 48.3% on the CityLayout-100 dataset, demonstrating the potential for translating visual observations into editable city code with both 2D and 3D representations.
Visual observations can be transformed into editable city layouts with a surprising level of detail and accuracy using a multimodal large language model.
Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.
