Interpretability & Mechanistic Interp - Weekly Roundup

Generative Verification (GenV) is introduced, which distills an offline Z3-equivalence oracle into a reference-free, continuous reference-equivalence score by repurposing the language model's native vocabulary space and theoretically proves that structural, verdict-only verification heuristics are mathematically bounded to chance-level detection on these deceptively valid traces.

The Explainability Assistant is introduced, an open-source conversational XAI system that leverages the function-calling capabilities of modern Large Language Models (LLMs) to overcome limitations and achieves 94% intent-parsing accuracy, supports flexible natural language interaction, and adapts to different ML problem types without task-specific fine-tuning.

This framework introduces kernel-based output representations that enable time-dependency-aware explanations at multiple levels of temporal granularity, including time-specific, time-resolved, and time-aggregated, while providing a unified view in which existing methods arise as special cases.

Where recent studies report that probe-detected errors are resistant to interventions, it is found that in-context binding is a setting in which probes are actionable.

A single classical oracle is constructed relative to which one-way puzzles do not exist, while an EFI pair survives every distinguisher that queries the oracle classically throughout and holds advice about it, making its one superposition query at the end.

Generative Verification (GenV) is introduced, which distills an offline Z3-equivalence oracle into a reference-free, continuous reference-equivalence score by repurposing the language model's native vocabulary space and theoretically proves that structural, verdict-only verification heuristics are mathematically bounded to chance-level detection on these deceptively valid traces.

The Explainability Assistant is introduced, an open-source conversational XAI system that leverages the function-calling capabilities of modern Large Language Models (LLMs) to overcome limitations and achieves 94% intent-parsing accuracy, supports flexible natural language interaction, and adapts to different ML problem types without task-specific fine-tuning.

This framework introduces kernel-based output representations that enable time-dependency-aware explanations at multiple levels of temporal granularity, including time-specific, time-resolved, and time-aggregated, while providing a unified view in which existing methods arise as special cases.

Where recent studies report that probe-detected errors are resistant to interventions, it is found that in-context binding is a setting in which probes are actionable.

A single classical oracle is constructed relative to which one-way puzzles do not exist, while an EFI pair survives every distinguisher that queries the oracle classically throughout and holds advice about it, making its one superposition query at the end.

Environment probing is introduced, a deployment-compatible extension that gives an existing asynchronous curator agent least-privilege, read-only world tools to check, scope, and refresh candidate memories and turns existing agent-memory curation into an environment-informed, auditable process while preserving a compact task-time interface.

Bridging the gap between theoretical optimality and practical representation learning, this work designs an implementation that imposes a translational bias on counterfactual trajectories - a constraint that aligns with how many concepts geometrically manifest in modern language models.

Digital function-oriented communication is presented, in which finite-alphabet symbol representations and receiver decisions are jointly designed so that multiple-access superposition encodes the desired function without recovering individual inputs.

Predicting blood-brain barrier permeability does not require complex graph neural networks: fifteen standard physicochemical descriptors paired with an optimized XGBoost model achieve a 0.928 ROC-AUC and 97.8% recall with full SHAP-derived mechanistic transparency.

Removing an architectural prior abruptly tanks Transformer recall from 77% to 9%, but smoothly annealing it away allows the model to permanently consolidate the underlying circuit.

Standard gradient-trained SAEs can achieve near-perfect reconstruction while remaining entirely misaligned with ground-truth features, reliably trapping models in an uninterpretable "diffuse phase" rather than reaching theoretical sparse minima.

Across models from 124M to 7B parameters, token predictions collapse to a scale-invariant core of just 1% to 3% of the network, enabling direct, closed-form model editing in single units without gradient descent.

Balanced $k$-shot sampling inadvertently breaks classical small-sample discriminant estimators through an exact algebraic degeneracy, yet even after deriving closed-form repairs, a standard cross-validated logistic probe remains superior on LLM embeddings.

Subtracting logit lenses across matched prompt pairs reveals internal circuits without activation patching, yet exposes a crucial interpretability trap: identical computational distinctions map to almost entirely disjoint token spaces across different random seeds.

The sparse autoencoder features optimal for document-topic mixture estimation actually undermine topic readability, revealing that monosemantic topic modeling requires decoupling inference from semantic labeling.

Shrinking the geometric modality gap in VLMs can paradoxically destroy retrieval performance, exposing a critical disconnect between representation geometry and task alignment that unbalanced optimal transport tracks with near-perfect fidelity (0.973 vs. 0.392 Spearman).

A reference-based method that audits bias in hidden-state representations across related model variants across related model variants, for example before and after fine-tuning, which is complementary to output-based auditing rather than a replacement for it.

Weight-space model editing no longer requires fine-tuning, as forward-pass activation vectors can be analytically converted into composable rank-one weight edits for precise behavioral control.

Reasoning traces with identical final answers and identical entropy curves can conceal radically different internal dynamics, from protracted multi-hypothesis competition to sudden late-stage revision.

Length generalization in transformers does not require double-exponential sample sizes: viewing inputs as compressed strings via power words slashes worst-case sample complexity down to polynomial.

Automated benchmarks for explainability are fundamentally brittle: LLM "simulators" consistently game the metric by solving the task directly through semantic priors or exploiting label leakage rather than actually relying on the explanations.

GraphFAS (Graph Feature Automated Selection), a distributed feature selection procedure based on Boruta that decouples feature aggregation from model training, enabling direct integration with tabular models and direct compatibility with TreeSHAPbased explanations.

Multi-class entanglement in deep representations is overwhelmingly pairwise rather than high-order, and data augmentation—not width, depth, or weight decay—is the sole training factor that genuinely separates class manifolds beyond chance.

Scaling and post-hoc interpretability hit a hard wall in agentic AI because models cannot recover causal invariances that observational interaction data never contained in the first place.

Identical ethical dilemmas trigger contradictory value judgments across languages in frontier LLMs, but inference-time steering vectors extracted from hidden-state discrepancies can reliably enforce culture-specific alignment without fine-tuning.

Where an LLM represents a stereotype is detached from where it acts on it: linear decodability peaks up to 53% of model depth earlier than causal attribution, while fewer than 18% of stereotype-associated SAE features transfer across languages.

Transformers collapse their latent space into a single dominant direction while SSMs spread information isotropically, yet both converge on identical effective capacity and local semantic manifolds.

An auditable decision record that carries four aligned cues into a late calibration step: a passive detector score, a conditional keyed-probe score on a marked derivative, retrieval support, and a speaker-profile margin, together with explicit disagreement coordinates is answered.

Steering vectors for distinct model behaviors remain naturally orthogonal in the residual stream, enabling training-free, multi-attribute control over language, safety, and style simply by stratifying injection across model depth.

Neural networks exhibit an internal "feeling of error"—manifesting as sharply suppressed concept activations inside Sparse Autoencoders—that reliably flags segmentation failures where standard output uncertainty metrics fail.

ViT weights harbor fully legible, input-invariant concept graphs of the world, unlocking targeted circuit interventions that improve shortcut removal by 11.0% over prior baselines.

Weakly supervised attention maps routinely latch onto spurious confounders—until you regularize them with the minimal attention shift required to flip the model's decision.

HiBRIDGE combines strong predictive performance with a structured decision process that supports more interpretable explanations of robot behaviour, and is demonstrated to be feasible for autonomous real-time group interaction.

Whether AI agents can act as scientists utilizing SAE tools for autonomous mechanistic discovery is evaluated to establish experimental model understanding as a measurable capability for closed-loop autonomous AI R&D.

Contrary to widespread assumptions, RoPE is not responsible for attention sinks and outlier activations; the true culprits are causal-mask self-concentration and value-non-mixing at the sequence start.

Satellite foundation models already encode the latent physical signals needed to map ground-level urban livability, bypassing the need for costly ground surveys in data-scarce regions.

Vision encoders encode an object's canonical color even when all color is stripped from the input image, exposing semantic priors that VLM post-training radically reorganizes.

Residual streams are dominated by transient feature proposals that adjacent layers immediately cancel out, upending the foundational assumption that transformer representations accumulate monotonically.

The topological geometry of an LLM's attention maps reveals whether code is hallucinated before a single line is executed, outperforming traditional pre-execution heuristics across models up to 34B parameters.

Detecting a backdoor trigger does not mean you can stop it: SAE features in attention and MLPs reliably flag trojan inputs but lack causal control, whereas sparse directions in the residual stream hold full steering power over backdoor execution.

Video models with identical segmentation accuracy can harbor severe internal explanation instability, as intermediate recurrent layers exhibit chaotic saliency jitter that conventional frame-wise XAI tools completely fail to diagnose.

Transformers encode an inferred partner's expertise early in the network, but interventions at peak decodability produce virtually zero downstream effect until patched past the midpoint.

CARDEA is a unified large vision-language model that serves as the inference core of a CAG pipeline that runs an end-to-end CAG pipeline from raw multi-view videos through keyframe selection to study-level diagnosis while exposing auditable spatial evidence behind its conclusions.

Instruction tuning degrades the latent moral geometry of LLMs, even though distribution-driven steering vectors can naturally mirror human value topologies to enable coherent cross-value generalization.