Search papers, labs, and topics across Lattice.
This paper introduces the Association Restoration Test (ART), a diagnostic tool designed to assess the functional usability of retained associations in classifiers after association unlearning. By estimating class-conditional association directions and amplifying residual components, ART evaluates whether learned shortcuts can still influence classifier outputs. The results across multiple datasets demonstrate that traditional output metrics and representation probes fail to capture the nuances of shortcut mitigation, highlighting the need for restoration-aware evaluation methods in AI systems.
Unlearning shortcuts doesn't guarantee their complete removal; ART reveals that some associations can still be functionally restored, challenging existing evaluation methods.
Association unlearning aims to disable learned label-attribute shortcuts while preserving task performance. Existing evaluations mainly measure output-level robustness or probe whether shortcut attributes remain readable in frozen features, but neither test determines whether a retained association remains functionally usable by the original classifier. We propose the Association Restoration Test (ART), a post-hoc diagnostic for functional shortcut restorability. ART estimates class-conditional association directions, amplifies residual components, and evaluates the modified features with the original classifier head. Across Waterbirds, CelebA, SpuCoDogs, and an ISIC timestamp-artifact extension, we show that output metrics, representation probes, and ART characterize distinct aspects of shortcut mitigation. These findings motivate restoration-aware evaluation for unlearning and shortcut-mitigation methods that target learned associations rather than individual classes or concepts.
Stanford HAI