Search papers, labs, and topics across Lattice.
We introduce HyperBrowseComp, a multilingual and multimodal benchmark with 423 challenging, human-validated questions across 13 languages, designed to stress-test web-browsing agents by requiring the discovery of obscure evidence and multi-step reasoning over heterogeneous sources like videos and scanned documents. By filtering out questions answerable by parametric knowledge alone, this benchmark specifically targets agents' ability to perform persistent information seeking on the open web. Evaluation of several models reveals significant performance gaps, highlighting the need for more robust agents capable of complex evidence retrieval and synthesis.
Most web-browsing LLM agents struggle to find obscure, multi-hop information across diverse modalities and languages, even when explicitly designed for web interaction.
We introduce HyperBrowseComp, a multilingual and multimodal browsing benchmark comprising 423 manually authored and human-validated questions across 13 languages, written by native or highly proficient speakers. Questions are designed to be extremely challenging. Each question targets a concise, publicly verifiable answer whose discovery requires locating obscure evidence, following multi-step clue chains, or inspecting heterogeneous sources such as videos, scanned documents, images, or maps. Easier questions are filtered out by evaluating them with models without internet access to reduce the likelihood that they can be answered with parametric knowledge alone. We evaluate several models using provider-native search and a shared external retrieval harness under a common agent protocol. To contextualize model performance and effort, we also conduct a human evaluation on a sample of the questions. HyperBrowseComp provides a challenging testbed for persistent information seeking across languages and evidence modalities, with difficulty arising from discovering and connecting evidence on the open web.
