Search papers, labs, and topics across Lattice.
This paper introduces counterfactual transition graphs (CTGs) as a novel framework for evaluating the structural relationships between classes in time-series classifiers, moving beyond traditional single-example evaluations. By analyzing a six-class hand-movement task, the authors reveal a complex topology of class transitions that contradicts expectations based on classifier accuracy, indicating that the most confidently separated boundaries are the least traversable by in-distribution edits. The findings highlight significant discrepancies between counterfactual reachability and classifier performance, suggesting that current evaluation methods may overlook critical interpretability insights.
Counterfactual reachability reveals surprising disconnects with classifier accuracy, challenging conventional wisdom about model boundaries.
Counterfactual (CF) explanations for time-series classifiers are usually evaluated one example at a time: what minimal edit flips this single window's prediction? We argue that the more informative question for diagnostic interpretability is structural: how does the classifier connect its own classes to each other? We propose a counterfactual transition graph (CGT) in which each node is a class and each edge weight is the CF reliability of the transition from one prototype to another under a proximity aware retrieval sweep. On a six-class hand-movement task, we induce a CGT that reveals a non-trivial topology, which is not predicted by the binary confusion matrix: it shows that counterfactual reachability does not align with classifier accuracy and even runs counter to it (Spearman $蟻=-0.37$ over the 15 pairs), i.e. the boundaries the classifier separates most confidently are among those an in-distribution edit can least often cross. Our framework is method agnostic, i.e. any CF-explainers can be used. Presently, we use it to juxtapose replacement-based CFs with gradient-based CFs; gradient-based methods reach almost any class by stepping off the data manifold, while replacement-based methods stay on it and fail on precisely the rigid boundaries.
