Search papers, labs, and topics across Lattice.
This study investigates how ideological biases in Retrieval-Augmented Generation (RAG) frameworks affect the outputs of large language models (LLMs), specifically focusing on the role of sampling temperature. By applying Lexical Multidimensional Analysis (LMDA) to a corpus of COVID-19 treatment articles, the authors identified three ideological discourses and assessed LLM responses to ideological questions at varying temperatures. The results reveal that moderate sampling temperatures enhance the alignment of generated answers with ideological reference texts, while low temperatures suppress this transfer, highlighting the nuanced interplay between retrieval grounding and ideological discourse in LLM outputs.
Ideological biases in RAG can significantly shape LLM outputs, with moderate sampling temperatures amplifying discourse alignment.
Retrieval-Augmented Generation (RAG) has been increasingly adopted to reduce hallucinations and strengthen the factual grounding of large language models (LLMs). While robustness to errors in the retrieval process has been explored, the impact of ideological bias on LLM outputs has been overlooked. For instance, if the retrieved material contains ideological positions, the RAG may transmit, amplify, or suppress such ideological discourses in its outputs. In this study, we address this issue by examining the influence of the RAG framework, comprising ideological discourses, in LLM-generated answers. To this end, we applied Lexical Multidimensional Analysis (LMDA) on a corpus of 1,117 COVID-19 treatment articles, identifying three ideological discourses. This corpus is then used as the external knowledge source for the RAG. We assessed several LLMs by having the models answer ideological questions at different sampling temperatures. The generated texts were assessed semantically and lexically based on their similarities with ideological reference texts. Our findings show that the RAG framework is prone to transferring ideological discourses into LLM responses, with sampling temperature having a measurable impact on the strength of this transfer. Discoursive alignment between generated answers and the reference text is highest at moderate temperatures, where models balance stochasticity with retrieval grounding, and drops at low temperatures, indicating that overly deterministic sampling suppresses discourse transfer.
