Search papers, labs, and topics across Lattice.
This study investigates the performance of confidence-based model cascades for image classification under various data perturbations, revealing their vulnerability to input degradations that can disrupt routing decisions. The authors identify three critical failure modes that affect the cascades' ability to maintain accuracy and energy efficiency when faced with static and sequential perturbations. Their findings highlight the necessity of evaluating model cascades not only on clean data but also in terms of routing reliability under changing input distributions, achieving up to a 10-fold reduction in CO₂ emissions while maintaining competitive predictive performance.
Energy-efficient model cascades can falter under data perturbations, revealing hidden vulnerabilities that could undermine their effectiveness in real-world applications.
Prediction cascades significantly reduce energy consumption of Artificial Intelligence (AI) models while maintaining high predictive performance. The idea is that easy inputs are routed through a lightweight small model, and difficult uncertain cases are deferred to a larger model. While this design can improve computational efficiency on clean data, its effectiveness depends on the reliability of confidence-based routing. Input degradations, such as static corruptions and sequential perturbations, can shift model confidence and routing decisions. In this paper, we study confidence-based cascade frameworks for image classification and investigate how such degradations affect their confidence-based deferral behavior. We select a model cascade at the pareto-optimum of accuracy, routing quality, and energy consumption that achieves competitive predictive performance with an up to 10-fold decrease in CO$_2$ emissions. We study the behavior of that model cascade under input corruptions and analyze how the cascade's routing decisions change when the input distribution shifts. Our analysis identifies three failure modes. Static corruptions either (1) break the routing signal while the large model remains useful, or (2) degrade both models so deferral no longer recovers accuracy. Sequential perturbations reveal a third mode: predictions stabilize but deferral suppresses, yielding stable but unreliable predictions. These findings demonstrate that energy efficient model cascades require evaluation beyond clean accuracy, with explicit attention to routing reliability under distribution shift.
