Search papers, labs, and topics across Lattice.
This paper introduces GeoFidelity-Bench, a benchmark designed to evaluate the geographic fidelity of text-to-image models in generating street-view images based on specific road segments. The study reveals that while incorporating street and neighborhood names improves retrieval accuracy by 5.5 percentage points, the generated images still struggle to accurately represent specific segments, indicating a gap between local plausibility and exact identity. The findings underscore the limitations of current models in achieving segment-level fidelity, despite advancements in generating visually plausible urban environments.
Local names boost retrieval accuracy, but models still fail to generate images that faithfully represent specific street segments.
Text-to-image models can generate visually plausible city streets, but whether their outputs correspond to a requested road segment rather than a generic city prior remains unclear. We introduce GeoFidelity-Bench, a reference-panel benchmark for segment-conditioned geographic fidelity in street-view generation. It contains 7,117 curated Mapillary images covering 109 named OpenStreetMap road segments in 25 cities across six continents. For each generated panel, the benchmark ranks the target reference panel against panels from the nearest segment in the same city, other segments in the same city, and segments from other cities, making local discrimination rather than absolute target similarity the primary test. We evaluate six open-weight text-to-image generators under city-only, street-and-neighborhood, and GPS-augmented prompts. Adding street and neighborhood names is associated with an increase of 5.5 percentage points in top-1 retrieval accuracy over city-only prompts, with a 95% confidence interval from 3.4 to 7.7 percentage points. However, the similarity margin between the target and the nearest segment in the same city remains near zero, indicating that local names improve broad local plausibility more than exact segment identity. Prompts that keep the city fixed but use incorrect street or neighborhood names further show that only part of the gain depends on the correct local names, while appending raw GPS coordinates as ordinary text yields no statistically clear additional benefit. Held-out real-image queries successfully recover segment identity, showing that the curated references contain usable segment-level signal. GeoFidelity-Bench thus reveals a persistent gap between city- or neighborhood-plausible street-view generation and faithful generation for a specific road segment.
CMU ML