Search papers, labs, and topics across Lattice.
This paper introduces a multi-agent system that leverages open-source large language models (LLMs) to automate the detection and debunking of disinformation, addressing the inadequacies of manual verification in the face of widespread misinformation. By mimicking human decision-making processes and incorporating a consensus mechanism along with cognitive and knowledge diversity, the system outperforms individual LLMs like GPT-4 and GPT-3.5 in various disinformation detection tasks across multiple languages. The approach not only enhances efficiency but also promotes transparency through the use of accessible models, making it a significant advancement in the fight against disinformation.
A multi-agent system using open-source LLMs outperforms leading models in detecting disinformation by mimicking human cognitive processes.
In contemporary societies, the threat of disinformation has reached alarming levels, exacerbated by the proliferation of electronic communication, social media, and advancements in artificial intelligence. As a result, there is an urgent need to develop effective countermeasures to mitigate this menace. However, the sheer scale of the problem renders manual fact-checking and human-based verification inadequate, underscoring the necessity for automated methods to detect and debunk disinformation. This article proposes a novel approach based on a multi-agent system that emulates the decision-making processes of human annotators engaged in disinformation detection tasks. By incorporating a consensus mechanism, diversity in cognition and diversity in knowledge, and also hierarchical structure, inspired by human annotators' behavior, the proposed method achieves superior results compared to individual Large Language Models (LLMs), including GPT 4 and GPT 3.5. The system leverages open models (e.g., LLaMA, Kimi, Qwen, Deepseek and LLaMA-Nemotron) to ensure greater transparency. The evaluation of the proposed method encompasses datasets in languages with varying resource availability, including English (high-resource), Polish (medium-resource), Slovak (low-resource) and Bulgarian (low-resource). Experiments were conducted on tasks such as direct disinformation detection, identification of texts worthy of verification, and detection of texts containing verifiable factual claims.
