Search papers, labs, and topics across Lattice.
This paper addresses the vulnerabilities of retrieval-augmented generation (RAG) systems to knowledge corruption attacks, which can manipulate outputs by poisoning input documents. The authors introduce SecureCollaRAG, a Byzantine-tolerant framework that employs a Multi-source Knowledge Validation Mechanism to verify document provenance through dynamic GNN-based credibility scoring. Extensive evaluations reveal that SecureCollaRAG effectively prevents knowledge corruption while maintaining the integrity of domain knowledge, even under non-IID data distributions.
SecureCollaRAG can thwart knowledge corruption attacks in RAG systems, ensuring the integrity of generated outputs against adversarial manipulations.
While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisoning documents provided by RAG system to manipulate LLM outputs. To counter this threat, we propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework leveraging Multi-source Knowledge Validation Mechanism. Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring, effectively preventing stealthy knowledge corruption attacks while preserving essential domain knowledge integrity. Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions. Content warning: This paper contains unfiltered content generated by LLMs that may contain malicious contents.
