Search papers, labs, and topics across Lattice.
The paper introduces RIVA, a multi-agent system designed to detect configuration drift in infrastructure-as-code (IaC) environments, addressing the vulnerability of LLM-based agents to erroneous tool outputs. RIVA employs a verifier agent and a tool generation agent that iteratively cross-validate each other's outputs, leveraging multi-perspective verification and tool call history tracking to mitigate the impact of incorrect tool responses. Experiments on the AIOpsLab benchmark demonstrate that RIVA significantly improves task accuracy in the presence of erroneous tool responses, increasing it from 27.3% (baseline ReAct agent) to 50.0%, and also improves accuracy from 28% to 43.8% even without erroneous tool responses.
LLM agents can now detect configuration drift in cloud infrastructure more reliably, even when their tools return garbage.
Infrastructure as code (IaC) tools automate cloud provisioning but verifying that deployed systems remain consistent with the IaC specifications remains challenging. Such configuration drift occurs because of bugs in the IaC specification, manual changes, or system updates. Large language model (LLM)-based agentic AI systems can automate the analysis of large volumes of telemetry data, making them suitable for the detection of configuration drift. However, existing agentic systems implicitly assume that the tools they invoke always return correct outputs, making them vulnerable to erroneous tool responses. Since agents cannot distinguish whether an anomalous tool output reflects a real infrastructure problem or a broken tool, such errors may cause missed drift or false alarms, reducing reliability precisely when it is most needed. We introduce RIVA (Robust Infrastructure by Verification Agents), a novel multi-agent system that performs robust IaC verification even when tools produce incorrect or misleading outputs. RIVA employs two specialized agents, a verifier agent and a tool generation agent, that collaborate through iterative cross-validation, multi-perspective verification, and tool call history tracking. Evaluation on the AIOpsLab benchmark demonstrates that RIVA, in the presence of erroneous tool responses, recovers task accuracy from 27.3% when using a baseline ReAct agent to 50.0% on average. RIVA also improves task accuracy 28% to 43.8% without erroneous tool responses. Our results show that cross-validation of diverse tool calls enables more reliable autonomous infrastructure verification in production cloud environments.
