Search papers, labs, and topics across Lattice.
This study evaluates the trustworthiness of sources in an AI information service, Evr贸puvefur, which combines a curated knowledge base with live web search to answer questions about the European Union. Expert evaluations revealed that while web search provided broader coverage, it compromised source quality, with 35% of web-sourced answers flagged as untrustworthy, compared to minimal issues with curated sources. The findings highlight a critical trade-off between coverage and trustworthiness in AI-generated responses, emphasizing the need for transparency in public AI services.
Trustworthiness in AI-generated responses is often overlooked, yet this study reveals that web search can lead to a staggering 35% of answers citing unreliable sources.
Public institutions increasingly use large language models (LLMs) to answer citizens'questions, often pairing a curated knowledge base with live web search, yet whether the sources behind these answers can be trusted has received little empirical scrutiny. We report a pre-launch expert evaluation of Evr\'opuvefur, an independent, government-funded service run by the University of Iceland that answers questions about the European Union, conducted as Iceland prepared for its referendum of 29 August 2026 on whether to resume EU accession talks. Five domain experts produced 551 evaluations of 449 AI-generated answers, scoring each against a seven-criterion quality rubric and, separately, flagging individual cited sources. We compared two retrieval paths: a curated local corpus (RAG) and open web search. In more than a third of the reviewed web-search answers (35%, 65 of 187), at least one cited source was flagged, almost always as untrustworthy or irrelevant; curated sources were flagged far less often and only for being out of date. Web search answered more questions, but at the cost of source quality; the curated corpus was trustworthy yet limited in coverage, and the model declined to respond when it fell short. The citation mix also passed over strong sources: across all 287 web-search answers, the system never cited R\'UV, the public broadcaster and the country's most widely used news source. A companion prompt ablation shows how weak prompt-level steering is: a trusted-domain list in the system prompt raised the share of citations to listed domains only from 12% to 21%. Fluency and topical fit did not predict source trustworthiness. We argue that source trustworthiness is a measurable yet largely invisible dimension of information quality in public AI services, and we discuss transparency-oriented responses and their trade-offs.
