Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of Vision-Language Models (VLMs) in detecting geometry clipping anomalies in video game quality assurance by employing a custom exploration agent to gather visual data and an automatic annotation pipeline for frame-level labeling. The research benchmarks six recent VLMs under zero-shot prompting conditions and assesses their performance sensitivity to various prompt designs. Results indicate that while VLMs can identify visual cues related to clipping, they struggle with false positives in ambiguous scenarios, positioning Gemini-3.1-Flash as the most accurate and robust model for this task.
VLMs can identify geometry clipping in games, but they generate significant false positives, highlighting their limitations in standalone bug detection.
In this work, we study the use of Vision-Language Models (VLMs) for anomaly detection in an agent-driven game Quality Assurance (QA) pipeline focusing on geometry clipping. In this evaluation, a custom exploration agent navigates a game level to collect visual observations, while the automatic annotation pipeline provides frame-level clipping labels. This setup allows us to evaluate recent VLMs on a controlled anomaly detection task without manual annotation. We benchmark six recent VLMs (Gemini, GPT, Qwen, Gemma, Llama, and Ministral) under a zero-shot prompting setting and analyse their sensitivity to four prompt variants. Our results show that while the VLMs can capture visual cues associated with geometry clipping, they all produce substantial false positives on visually ambiguous frames such as near-contact geometry and partial occlusions. Gemini-3.1-Flash achieves the best overall accuracy and is the most robust to prompt variation, while open-source models exhibit large precision--recall swings depending on the prompt design. These findings suggest that current VLMs are best suited as high-recall candidate filters within multi-stage QA pipelines rather than as standalone bug detectors.
