Search papers, labs, and topics across Lattice.
This paper audits 12 existing flaw reporting systems for AI, identifying five key design challenges that hinder effective communication and coordination among stakeholders. To address these issues, the authors introduce FLARE-AI, an open-source reporting system that enhances interoperability and streamlines the reporting process through conditional logic and standardized information dissemination. The key result is that FLARE-AI significantly lowers barriers for reporting AI flaws, facilitating faster remediation across the ecosystem by enabling a single submission to reach multiple relevant parties.
FLARE-AI transforms the fragmented AI flaw reporting landscape by enabling a single report to reach multiple stakeholders, enhancing collaboration and speeding up remediation efforts.
Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety. Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what or where to report, and groups who receive reports rarely share them with other relevant stakeholders. As a result, good-faith reporters duplicate effort by submitting many different forms, and recipients lack standardized, triage-ready information. We audit 12 reporting systems published by AI developers, cybersecurity groups, and AI flaw aggregators, identifying five recurring design challenges spanning discoverability, scope, information collection, coordination, and guidance for strict-liability cases. Building on this analysis and feedback from 49 experts across 32 organizations representing developers, security researchers, and ecosystem coordinators, we introduce FLARE-AI, an open-source AI flaw reporting system designed for interoperability with existing systems. FLARE-AI streamlines flaw report creation by collecting triage-relevant information through conditional logic and early classification, then enables optional dissemination of standardized, machine-readable reports to multiple developers, coordinators, and incident registries from a single submission. By lowering barriers to reporting AI flaws and improving interoperability across stakeholders, FLARE-AI helps break down silos and accelerate remediation across the AI ecosystem.
Stanford HAI