Search papers, labs, and topics across Lattice.
This paper introduces a rate-distortion (RD) framework to analyze accuracy-robustness trade-offs in visual systems, treating stimulus-response behavior as a communication channel and deriving RD frontiers from confusion matrices. The framework summarizes systems using two geometric signatures, slope (β) and curvature (κ), which quantify the marginal cost and abruptness of these trade-offs. Applying this framework to human psychophysics and deep vision models under image perturbations reveals that humans exhibit smoother trade-offs compared to the steeper, more brittle regimes of deep networks, even at matched accuracy.
Humans navigate the accuracy-robustness tradeoff with a smoothness and flexibility that eludes even the most robust modern deep networks.
Generalization to novel visual conditions remains a central challenge for both human and machine vision, yet standard robustness metrics offer limited insight into how systems trade accuracy for robustness. We introduce a rate-distortion-theoretic framework that treats stimulus-response behavior as an effective communication channel, derives rate-distortion (RD) frontiers from confusion matrices, and summarizes each system with two interpretable geometric signatures - slope ($β$) and curvature ($κ$) - which capture the marginal cost and abruptness of accuracy-robustness trade-offs. Applying this framework to human psychophysics and 18 deep vision models under controlled image perturbations, we compare generalization geometry across model architectures and training regimes. We find that both biological and artificial systems follow a common lossy-compression principle but occupy systematically different regions of RD space. In particular, humans exhibit smoother, more flexible trade-offs, whereas modern deep networks operate in steeper and more brittle regimes even at matched accuracy. Across training regimes, robustness training induces systematic but dissociable shifts in beta/kappa, revealing cases where improved robustness or accuracy does not translate into more human-like generalization geometry. These results demonstrate that RD geometry provides a compact, model-agnostic lens for comparing generalization behavior across systems beyond standard accuracy-based metrics.
