Interpretability & Mechanistic Interp - Weekly Roundup

Transformers may succeed at time series forecasting without relying on the complex superposition that drives their power in NLP, challenging the assumption that these models are leveraging rich compositional representations.

Decoding driver behavior jumps from 73% to 81% accuracy by fusing EEG, EMG, and GSR signals, pinpointing the physiological markers that matter most.

Steering neural networks through the intrinsic geometry of their activations unlocks more natural and controllable behaviors than traditional linear interventions.

Forget expensive on-site inspections: this multimodal model uses assessor text and GIS data to accurately predict building energy performance, enabling scalable retrofit planning.

Steering LLMs with conceptors—soft projection matrices capturing the full semantic subspace—yields more robust control and enables Boolean logic for composing concepts, moving beyond the limitations of single-vector steering.

Transformers may succeed at time series forecasting without relying on the complex superposition that drives their power in NLP, challenging the assumption that these models are leveraging rich compositional representations.

Decoding driver behavior jumps from 73% to 81% accuracy by fusing EEG, EMG, and GSR signals, pinpointing the physiological markers that matter most.

Steering neural networks through the intrinsic geometry of their activations unlocks more natural and controllable behaviors than traditional linear interventions.

Forget expensive on-site inspections: this multimodal model uses assessor text and GIS data to accurately predict building energy performance, enabling scalable retrofit planning.

Steering LLMs with conceptors—soft projection matrices capturing the full semantic subspace—yields more robust control and enables Boolean logic for composing concepts, moving beyond the limitations of single-vector steering.

Geometric continuity in deep networks isn't just a byproduct of depth, but an actively sculpted property arising from the interplay of residual connections and symmetry-breaking activations.

Forget retraining: this model learns interpretable logical rules from data in a zero-shot manner by encoding literals with domain-agnostic statistical properties.

Feature importance in machine learning models can be surprisingly unreliable: even when models predict accurately, the features they deem important can vary wildly, especially with small datasets.

Token embedding geometry isn't just abstract math—it directly mirrors how language models internally represent and reason about the world, as shown by its alignment with board state and piece importance in chess.

Symmetric spectral analysis of attention is fundamentally blind to information flow direction, but a simple asymmetry coefficient can restore the signal.

LLMs can construct interpretable, multi-layered models of individual student cognition from journal entries, opening new possibilities for personalized education.

Forget opaque transformers: Gyan offers SOTA language modeling with full interpretability, lower compute, and human-like compositional understanding.

Attention heads hold the key to detecting LLM hallucinations, offering a lightweight, white-box alternative to expensive sampling or external models.

Ditch the black box: This unsupervised semantic projection method rivals supervised models in psychological assessment, offering interpretability and generalizability that supervised methods lack.

Stop reinventing the wheel (or worse, comparing apples to oranges) in XAI evaluation: a standardized "XAI Evaluation Card" could finally bring clarity and rigor to a fragmented field.

Stop squinting at Nsight Compute profiles: KEET uses LLMs to automatically diagnose GPU kernel bottlenecks and suggest optimizations in plain English.

Make your prompts 5x more interpretable without hurting accuracy: IPL combines discrete token selection with continuous optimization, and it's plug-and-play with existing methods.

Activation steering can finally match the nuanced control of prompt engineering: token-specific interventions learned from prompts let you steer LLMs more effectively.

Clinicians trust AI recommendations nearly 3x more when those recommendations are broken down into verifiable facts linked to source guidelines, blowing traditional explainability out of the water.

Forget human-readable models: Agentic-imodels evolves ML models that are optimized for LLM interpretability, boosting agentic data science performance by up to 73%.

Transformers generalize out-of-distribution not by clever interpolation, but by learning a separate, orthogonal representation subspace for unseen tasks.

Releasing differentially private explanations of GNN predictions doesn't hide your graph structure as much as you think: adversaries can reconstruct it with surprising accuracy.

Fixed confidence thresholds are holding back explainable autonomous driving systems, but this new adaptive approach and dataset could unlock better performance and cross-cultural understanding.

Adversarial attacks on speech models leave tell-tale geometric fingerprints in early representation layers that can be detected without transcripts.