Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study audits the citation practices of three AI-driven health information platforms (ChatGPT, Perplexity, Google AI Overview) across twenty mental health queries in English and six other languages. The analysis revealed that a small number of domains dominated citation counts, with the top ten sources accounting for 43.6% of citations, and highlighted significant differences in citation consistency and source type preferences among platforms. The findings underscore the need for improved source evaluation in conversational AI systems, especially as non-English queries received fewer citations and less appropriate resource routing.
A staggering 43.6% of AI-generated health citations come from just ten sources, raising critical questions about the diversity and reliability of information provided to users.
Online health information seeking is shifting from keyword search, where users consider a ranked list of links, to conversational systems that compose a single answer and curate its citations. Source evaluation therefore passes from user to platform, yet what these systems surface is poorly characterized. We audited three free consumer products (ChatGPT, Perplexity, Google AI Overview) on twenty English mental health questions under two prompt conditions, with a subset of three also translated into six further languages of varying resource tiers. We recorded 15,942 citations across 1,140 responses and 1,713 unique domains, then classified every citation with a nine-category organizational typology applied by a deterministic classifier validated against human coding. Citations were heavily concentrated: the ten most-cited domains accounted for 43.6% of English citations, and government, commercial health, and academic sources were closely matched at roughly 22% each. Platforms differed little in typical citation volume but sharply in consistency and in the source types they favored. Explicitly requesting sources shifted composition only modestly. Non-English queries surfaced fewer citations and were routed to language-appropriate resources at significantly lower rates. We release the typology, classifier, and annotated corpus as reusable instruments for auditing generative health search.