Search papers, labs, and topics across Lattice.
This paper introduces ChatImage, a novel system that transforms long-form answers generated by Large Language Models (LLMs) into interactive visual images, enhancing user navigation and inspection. By normalizing textual content into structured visual modules and applying vision grounding models, ChatImage allows users to interact with specific parts of an answer through clickable hotspots, thereby improving the accessibility of complex information. The system's effectiveness is validated through a 30-question benchmark, demonstrating significant improvements in interaction-loop completion and visual alignment.
Interactive visualizations of LLM outputs can transform how users engage with complex information, allowing for targeted exploration without the need to sift through dense text.
Large Language Models (LLMs) can produce detailed answers to complex queries, but these answers are typically presented as dense linear text, which makes fine-grained inspection, navigation, and return visits difficult. We present ChatImage, a system that converts long-form LLM answers into interactive visual images. Given a textual answer, ChatImage first normalizes its content into structured visual modules, plans a visual layout, and renders a coherent image. It then applies a second grounding pass to the rendered image with vision grounding models such as LocateAnything and MiMo-Vision, with optional SAM-style mask refinement, to identify the visible regions that should support interaction. From these grounded regions, ChatImage overlays transparent clickable hotspots on the image. Each hotspot opens a detail panel and a region-scoped follow-up thread, allowing the user to inspect and query a specific part of the answer without re-reading the full response. Instead of treating planned coordinates as the final interaction geometry, ChatImage uses them as priors and grounds the interaction targets after rendering, which improves consistency between visual content and clickable regions. We release a reference implementation and introduce a 30-question benchmark covering infographic, map, and scene-based answer formats. Evaluation with configured external models reports interaction-loop completion, a strict visual-alignment gate, and a SAM-based mask-completeness diagnostic.
