Search papers, labs, and topics across Lattice.
This paper analyzes the WildChat dataset to quantify and categorize "invisible failures" in human-AI interactions, where the AI underperforms without explicit user indication. They identify eight archetypes of these failures, revealing that 78% of AI failures are invisible and cluster into higher-level failure types. The analysis further suggests that 94% of interactional failures, which constitute 91% of all failures, would persist even with more capable AI models, highlighting the importance of addressing interactional dynamics.
Most AI failures aren't the spectacular kind, but silent breakdowns in interaction that will persist even as models get smarter.
AI systems fail silently far more often than they fail visibly. In a large-scale quantitative analysis of human-AI interactions from the WildChat dataset, we find that 78% of AI failures are invisible: something went wrong but the user gave no overt indication that there was a problem. These invisible failures cluster into eight archetypes that help us characterize where and how AI systems are failing to meet users' needs. In addition, the archetypes show systematic co-occurrence patterns indicating higher-level failure types. To address the question of whether these archetypes will remain relevant as AI systems become more capable, we also assess failures for whether they are primarily interactional or capability-driven, finding that 91% involve interactional dynamics, and we estimate that 94% of such failures would persist even with a more capable model. Finally, we illustrate how the archetypes help us to identify systematic and variable AI limitations across different usage domains. Overall, we argue that our invisible failure taxonomy can be a key component in reliable failure monitoring for product developers, scientists, and policy makers. Our code and data are available at https://github.com/bigspinai/bigspin-invisible-failure-archetypes
