Search papers, labs, and topics across Lattice.
This study investigates the effectiveness of retrieval-augmented generation (RAG) in addressing factual inaccuracies in large language models (LLMs) by analyzing their performance on a benchmark of approximately 2,000 public companies across various contexts. The findings reveal significant geographic disparities in accuracy, particularly in no-context scenarios, and show that while perfect context enhances performance, it does not eliminate these disparities, indicating a complex relationship between retrieval effectiveness and internal model representations. Additionally, the research highlights that larger models do not mitigate these structural biases, challenging the assumption that RAG uniformly corrects factual errors across diverse contexts.
Geographic biases in LLMs persist even with retrieval-augmented generation, revealing that larger models don鈥檛 necessarily solve the problem.
Retrieval-augmented generation (RAG) is widely assumed to mitigate factual errors in large language models (LLMs), but it remains unclear whether retrieval uniformly compensates for missing knowledge. We study this question in a controlled factual QA setting over public companies, constructing a benchmark of approximately 2,000 firms across global equity indices. We evaluate six LLMs on four atomic attributes under four conditions: no-context, perfect context, misleading context, and distraction context. We find strong geographic disparities in no-context accuracy, indicating uneven parametric knowledge. While perfect context improves performance, it does not eliminate these gaps: gains are correlated with baseline accuracy, suggesting retrieval effectiveness is coupled to internal representations. Under misleading context, models frequently copy incorrect information. Larger models improve overall performance but do not remove these structural effects. These results challenge the view of RAG as a universal corrective and highlight the interaction between model knowledge, context quality, and entity representation.
