Search papers, labs, and topics across Lattice.
This paper introduces a framework for evaluating and improving the auditability of LLM agents, arguing that auditability is a prerequisite for accountability. They define five dimensions of agent auditability and three mechanism classes (detect, enforce, recover) to achieve it. Through ecosystem measurements, runtime feasibility analysis, and recovery experiments, the authors demonstrate the current lack of auditability in existing systems and propose an Auditability Card to guide future development.
You can add tamper-evident logging to LLM agents for just 8ms, unlocking accountability even when standard logs fail.
LLM agents call tools, query databases, delegate tasks, and trigger external side effects. Once an agent system can act in the world, the question is no longer only whether harmful actions can be prevented--it is whether those actions remain answerable after deployment. We distinguish accountability (the ability to determine compliance and assign responsibility), auditability (the system property that makes accountability possible), and auditing (the process of reconstructing behavior from trustworthy evidence). Our claim is direct: no agent system can be accountable without auditability. To make this operational, we define five dimensions of agent auditability, i.e., action recoverability, lifecycle coverage, policy checkability, responsibility attribution, and evidence integrity, and identify three mechanism classes (detect, enforce, recover) whose temporal information-and-intervention constraints explain why, in practice, no single approach suffices. We support the position with layered evidence rather than a single benchmark: lower-bound ecosystem measurements suggest that even basic security prerequisites for auditability are widely unmet (617 security findings across six prominent open-source projects); runtime feasibility results show that pre-execution mediation with tamper-evident records adds only 8.3 ms median overhead; and controlled recovery experiments show that responsibility-relevant information can be partially recovered even when conventional logs are missing. We propose an Auditability Card for agent systems and identify six open research problems organized by mechanism class.
