Search papers, labs, and topics across Lattice.
To prevent silent, clinically hazardous failures in cancer segmentation, the authors investigate whether vision models encode an internal latent signature of their own errors analogous to a human "feeling of error." Applying Sparse Autoencoders (SAEs) to hidden activations across prostate, pancreatic, and brain cancer models, they discover that segmentation failures reliably exhibit a distinct latent signature characterized by fewer active concepts and suppressed activation magnitudes. Classifiers trained directly on these SAE concept activations detect failures more accurately than traditional output-based uncertainty metrics while providing human-interpretable explanations without degrading segmentation quality.
Neural networks exhibit an internal "feeling of error"鈥攎anifesting as sharply suppressed concept activations inside Sparse Autoencoders鈥攖hat reliably flags segmentation failures where standard output uncertainty metrics fail.
Cancer segmentation models can fail silently, generating plausible but incorrect masks that risk missed findings or unnecessary biopsies. A critical question arises: Do AI models "know" when they are wrong, and if so, can we use the signal to predict their own failures? Humans do have a "Feeling of Error" (FOE): a spontaneous sense of unease that flags a potential error during thinking. We investigate whether cancer segmentation models exhibit an analogous internal signal. Unlike output-level cues (e.g., prediction confidence or uncertainty), which offer no insight into why a failure occurs and suffer from a sensitivity-quality tradeoff where high detection sensitivity could degrade overall segmentation quality. We instead propose to capture the model's FOE from its inner workings. Using mechanistic interpretability tools, specifically Sparse Autoencoders, we decompose internal neural activations into a dictionary of human-interpretable concepts and show that failure cases exhibit a distinct latent signature: fewer active concepts with lower activation magnitudes compared to successful segmentation. By training a classifier on these concept activations, we achieve accurate failure detection along with explanations for the model's mistakes. Experiments on prostate, pancreatic, and brain cancer segmentation demonstrate that our approach outperforms output-based methods in failure detection while preserving segmentation quality.
