Search papers, labs, and topics across Lattice.
This study introduces Traccia, a governance platform built on OpenTelemetry that addresses the shortcomings of existing AI evaluation and monitoring systems, particularly in the context of compliance with the EU's AI Act. By integrating telemetry data and semantic assessments into a hashed trace ledger, Traccia enhances transparency and accountability while safeguarding against alignment drift and unauthorized AI deployments. The platform automatically generates compliance evidence packages that meet regulatory standards without compromising data privacy, thereby providing a robust framework for managing autonomous AI systems.
Traccia transforms AI governance by seamlessly integrating compliance tracking into the operational fabric of autonomous systems, ensuring accountability without sacrificing privacy.
The rapid development of Large Language Models (LLMs) and Artificial Intelligent (AI) powered autonomous agents has fundamentally changed the existing forms of software governance. In spite of the rigorous standards of transparency and account ability required according to the international frameworks such as the European Union's AI Act, there is a considerable gap between theory and reality. The present study discusses the inherent drawbacks of currently utilized platforms for LLM evaluation, machine learning workflow, and application performance monitoring in general. It has been shown that current disjointed solutions fail to protect unbound state space agentic architecture from serious threats such as alignment drift, SaaS security concerns, and unauthorized deployment of shadow AI systems. Moreover, a solution is proposed for overcoming the discussed challenges in form of a coherent multi-level AI governance stack Traccia built on the top of OpenTelemetry infrastructure platform. Traccia resolves the last mile for AI Alignment by adding the telemetry data, passive semantic guardrail assessment, and execution lineage into a hashed trace ledger. Traccia automatically creates compliance evidence packages by appending tamper-resistant fingerprints and SHA-256 content hash, that map to regulatory requirements (Articles 12, 14, 19, 26(6), and 50 of the EU AI Act) without invading any data privacy. By performing this evaluation in a methodical manner, a solid machine-readable base has been created for enterprise-wide management of autonomous AI systems.
