Interpretability & Mechanistic Interp - Weekly Roundup

Most observed IPv6 Fragment-EH patterns are simple within-packet co-occurrences, not complex temporal network rules, a finding rigorously established by a novel negative-control protocol.

Transformer weight-space learning is now invariant to the full, rich symmetry group of attention heads and token embeddings.

Prompt adaptation methods don't just change what LLMs say, they fundamentally rewire *how* they compute, with distinct mechanisms for text optimization, prompt tuning, and prefix tuning.

Explanations from Natural Language Autoencoders can identify their target activation but fail to discriminate it from plausible rivals, suggesting current training paradigms may not yield robust oversight tools.

A grid-based world model can achieve 89% out-of-view object recall and outperform a Kalman tracker in view, but a node-based approach offers drastically superior memory efficiency and robustness to unmodeled velocities.

Transformer weight-space learning is now invariant to the full, rich symmetry group of attention heads and token embeddings.

Prompt adaptation methods don't just change what LLMs say, they fundamentally rewire *how* they compute, with distinct mechanisms for text optimization, prompt tuning, and prefix tuning.

Explanations from Natural Language Autoencoders can identify their target activation but fail to discriminate it from plausible rivals, suggesting current training paradigms may not yield robust oversight tools.

A grid-based world model can achieve 89% out-of-view object recall and outperform a Kalman tracker in view, but a node-based approach offers drastically superior memory efficiency and robustness to unmodeled velocities.

Pruning LLMs while preserving their influence functions is now over 28x faster and more effective, unlocking better data selection for downstream tasks.

A simple post-hoc geometric calibration method recovers lost out-of-domain capabilities and boosts in-domain performance after SFT, even on large, diverse models.

Weak interactive feedback can dynamically steer LLMs to the correct similarity criterion for test-time decisions, unlocking significant performance gains beyond static mixtures.

Sparse autoencoders recover known factors 80% better by training in the data's generative space, not its scaled readout space.

LLM judges can be recalibrated to align with human preferences by probing their sensitivity to evaluative biases, even without explicit rubric criteria.

Adam's implicit bias offers a practical defense against training data reconstruction attacks, unlike SGD and Muon which leave reconstructible fingerprints.

Multilingual LMs encode a universal color-language geometry, but not a universal color-perception geometry, with perceptual grounding fading in later layers.

LLM decoder states exhibit distinct informational and geometric compartments, enabling principled layer selection for speculative decoding that surpasses heuristic approaches.

MLLMs possess an intrinsic "S-Space" that acts as a spatial workspace, encoding object coordinates and enabling viewpoint transformations, even under non-spatial queries.

Source-side training dynamics, not just current values or training time, prospectively predict OOD degradation, even across model architectures.

Standard Shapley values are brittle; interval-valued Shapley values reveal their true robustness landscape in tree models.

A score-based method accelerates cluster DAG learning from interventions by two orders of magnitude, matching state-of-the-art edge recovery even on dense graphs.

Unifying factual support, counterfactual sensitivity, and exemplar tolerance reveals how GNNs truly reason about molecular properties, not just predict them.

Interpretable archetypes emerge from discrete data using a principled likelihood framework, unlocking insights in genomics and microbiome analysis previously inaccessible to standard AA.

Dynamic critic pessimism, guided by geometric disagreement, unlocks superior locomotion learning and efficiency in off-policy actor-critic methods.

Spectral analysis reveals that increasing latent dimensions in Koopman autoencoders improves dynamics learning by concentrating learned eigenvalues in low-residual regions, not just by reducing one-step prediction error.

Probes can now detect subtle, unverbalized LLM deception with near-perfect accuracy, even when the model's true intentions are hidden from external observation.

LLMs can be steered to reliably interpret complex genomic variants by grounding their reasoning in deterministic biological computations and evidence verification, avoiding hallucinated TF binding changes and fabricated experimental support.

Neural networks can learn interpretable, polytope-based latent representations that preserve performance and enable direct VQ training.

LLMs can attribute responsibility and blame, but their reasoning is incomplete and varies with model scale, with specific attribution dimensions demonstrably encoded in their latent spaces.

A 10x increase in AI time horizon is significantly easier for AI development in the 2-30 minute task range than for tasks outside this sweet spot.

Action chunking unlocks temporally interpretable decision trees that match neural network performance with up to 80% fewer parameters.

Most recalibration methods fail to guarantee calibration for critical outcomes, but this new approach ensures predictions are trustworthy even when focusing on rare or extreme events.

Most gradient-based methods fail to reliably attribute learned behaviors in online RL, with simple fluency metrics often proving more predictive than complex attribution techniques.

Input-blind controls can account for substantial oracle headroom in adaptive computation, challenging the assumption that observed gains directly reflect the efficacy of selected layer programs.

Later Transformer layers in tabular foundation models aren't always the smartest; revisiting earlier thoughts dramatically boosts performance.

Intrusion detectors confidently misclassify unseen attacks, but PairAudit's relational graph tokens uncover these overlooked errors, boosting human review accuracy by 30% on novel threats.

Gaussian regularizers like SIGReg can certify InfoNCE's optimal solutions but diverge significantly in their behavior away from the optimum, especially regarding nuisance channels and style retention.

AI malfunctions, like Clever Hans effects, can be directly traced to specific artifact patterns in the input data using a model-agnostic dictionary of interpretable concepts.

LLM-generated rules, not full trees, unlock efficient and interpretable few-shot tabular classification.

Most alignment model organisms degrade core capabilities and naturalness during training, undermining interpretability findings; a new multi-objective approach preserves these crucial aspects.

A GP's ability to fit data is predictable from its input design alone, revealing potential contradictions before any responses are observed.

Spurious forgetting in LLMs is a reversible access loss, not permanent knowledge erasure, and can be mitigated by decoupling shared representational shifts from fact-specific updates.

Generalization-heavy models are demonstrably more fragile during unlearning, suffering significantly higher performance degradation on retained data.

Conformal prediction set size reduction is formally justified as an information gain metric, bridging uncertainty quantification with classical information theory.

Diffusion models reveal feature generation order quantitatively, exposing architectural differences and suggesting ordered training benefits.

Training-free TFMs can outperform trained GCNs in calibration for node classification, and a novel homophily-gated approach significantly shrinks prediction sets without sacrificing validity.

Activation steering debiasing methods reduce LLM bias not by correcting underlying preferences, but by making the model less confident and more likely to abstain from answering.

Discovering task-specific, compact concept vocabularies from unsupervised pools unlocks inspectable AI without the need for manual annotations or massive, static concept sets.

A novel explainable power modeling framework achieves state-of-the-art prediction accuracy and generalization on diverse workloads with negligible hardware overhead, unlike opaque black-box models.

Unfrozen video-language models implicitly know when they've seen enough evidence, a signal that can be extracted and used to boost answer accuracy by nearly 10% without retraining.

Counterfactual explanations can now achieve near-perfect numerical sparsity and proximity by treating feature changes as a sparse transport problem solved with flow matching.

Latent signals for unanswerability generalize robustly across diverse question-answering datasets, but their utility in multi-turn dialogue hinges on capturing conversational dynamics rather than just static missing information.

Connectome-based GNNs achieve high accuracy but are dangerously overconfident, with misclassified patients receiving up to 95% confidence.

Tabular foundation models exhibit joint label processing, deviating from fixed-weight in-context learning and suggesting context labels dynamically alter each other's influence on predictions.

A small, causally identified set of attention heads reliably detects network infrastructure information across diverse LLMs, but their responsibility may reside in single neurons or be distributed across the head depending on the model.

LLMs can decode partner intents but struggle to translate this knowledge into cooperative actions, with explicit action recommendations proving far more effective than abstract rule statements.

Most observed IPv6 Fragment-EH patterns are simple within-packet co-occurrences, not complex temporal network rules, a finding rigorously established by a novel negative-control protocol.

Time-series foundation models, despite their promise for "what-if" analysis, exhibit attenuated dynamics and memorylessness, requiring synthetic fine-tuning to recover accurate causal inference.

Unlocking interpretable uncertainty in LLMs by mapping semantic doubt and certainty anchors to a low-dimensional subspace, outperforming baselines even when controlling for output length.