Search papers, labs, and topics across Lattice.
44 papers published across 1 lab.
RL gains in LLMs are surprisingly localized to a low-dimensional, non-infinitely compressible manifold, with control directions orthogonal to principal activation components.
Humans can directly control complex embodied agents using a handful of learned, composable skills discovered through self-play, bypassing the need for low-level action control.
Property-specific probing unlocks efficient, confidential multi-group fairness auditing, revealing bias structure without model reconstruction and resolving open sample complexity questions.
A single model's representation can predict how it disagrees with others, unlocking efficient input reliability estimation without needing multiple models at inference.
A single learned linear layer, not the shared Transformer weights, dictates distinct computations within each recurrent loop of a Looped Transformer, with attention patterns causally routing these operations.
Humans can directly control complex embodied agents using a handful of learned, composable skills discovered through self-play, bypassing the need for low-level action control.
Property-specific probing unlocks efficient, confidential multi-group fairness auditing, revealing bias structure without model reconstruction and resolving open sample complexity questions.
A single model's representation can predict how it disagrees with others, unlocking efficient input reliability estimation without needing multiple models at inference.
A single learned linear layer, not the shared Transformer weights, dictates distinct computations within each recurrent loop of a Looped Transformer, with attention patterns causally routing these operations.
Most LLM lie detectors fail when models adopt anti-factual personas, instead latching onto spurious correlations like instruction following.
The Green's operator reveals how computation is globally routed and reused across tasks and time in recurrent networks, offering a new lens beyond activity geometry.
Legal LLMs can now be optimized for nuanced quality beyond correctness, with rubrics that dissect response evaluation into Style, Element, and Chain of reasoning.
AI-driven music comparison tools can effectively surface subtle, expert-validated differences by letting users precisely scope comparisons and inspect AI-interpreted features.
Weight decay, not just learning rate, dictates the peak magnitude and lifespan of massive activations in transformers.
Auditing just 5% of token positions catches nearly all model threats—and simple chat structure picks those tokens better than internal computational signals without requiring a forward pass.
Generative models can only rely on efficient local denoising up to the exact moment they commit to global semantics, after which both phenomena collapse into a single sharp phase transition as system size grows.
Context expansion inflates an LLM's active token budget not to retrieve new facts, but simply to counteract softmax mass dilution caused by distractor token competition.
Far from merely clustering tokens, full Transformers act as pattern-forming dynamical systems that natively amplify traveling and rotating waves—an inductive bias that can be engineered at initialization to accelerate training.
Training data attribution on 32B-parameter models no longer requires intractable gradient storage: extreme 1-bit eigenbasis projection achieves competitive influence estimation accuracy at over 100× lower per-example footprint than standard baselines.
Most directions in a transformer's residual stream are functionally invisible to the terminal loss, exposing a thin observable subspace whose pullback Fisher geometry consistently outperforms standard attention weights for compression and distillation.
Key contribution not extracted.
Where an LLM stores a fact is rarely where you should erase it: high-precision storage localization picks the optimal unlearning intervention on just 17 of 36 targets, exposing a core blind spot in mechanistic model editing.
Models don't need to manipulate curved manifolds to compute over them: Qwen executes number comparison via simple linear projections and localized MLP boundaries, largely ignoring the non-linear manifold geometry of its own representations.
Even after 50,000 shared optimization steps and near-identical accuracy, neural networks preserve permanent internal footprints of their earliest training curricula.
Early transformer layers act as in-context PCA engines, compressing high-dimensional visual demonstrations into a universal, low-dimensional subspace where classification becomes trivially separable.
Swapping deep networks for symbolic logic rules specifically at taxonomic leaf nodes delivers verifiable, human-readable explanations across hundreds of granular classes without sacrificing hierarchical context.
Existing "private" graph explanations quietly leak sensitive edges with an empirical AUC of 1.0 and implied privacy budgets exceeding $\varepsilon = 700$, exposing a structural flaw that can only be solved by enforcing data-independent intervention universes.
Parameter matrices are no longer inscrutable black boxes: decomposing weights directly into the activation features they read and write enables scalable, causally editable circuit discovery at 8B scale.
A single trained neural network can harbor vastly different internal computations and representations within its local solution space, a diversity standard optimization misses.
On-policy LLM post-training's generalization advantage stems from its dynamic parameter update direction, a principle transferable to SFT for improved performance and efficiency.
Weight updates leave readable textual imprints: decoding parameter shifts into natural language enables direct, gradient-based behavioral steering without any downstream fine-tuning data.
Internal activation steering cannot displace standard alignment at scale, but lightweight internal probes paired with runtime interventions rescue models from the safety degradation caused by downstream fine-tuning at near-zero marginal inference cost.
Standard mechanistic circuits can mimic over 97% of a model's correct outputs while missing up to 89% of its errors, exposing that common ablation validations reward functional mimicry rather than true causal mechanisms.
Transformers actively protect their massive activation outliers through an upstream "read-blindness," where layers continually write to extreme residual coordinates while systematically refusing to read them.
Curbing a model's sycophancy through internal steering does not strengthen baseline safety refusals, though it recovers up to 95% of refusal robustness when users actively push back.
While Chain-of-Thought models overfit to local heuristics and collapse on longer reasoning chains, latent reasoning architectures autonomously discover a sparse, recurrent forward-search circuit that generalizes to unseen depths.
LLMs perform semantic recall via human-like explore-exploit foraging dynamics, depleting local concept representations and activating abstract category tokens right before strategically switching conceptual clusters.
Even without denoising objectives, transformers spontaneously self-repair corrupted inputs via a two-phase residual stream mechanism—and whether that repair succeeds is predictable from the very first transformer layer.
Alignment via DPO inadvertently creates functional valence: models systematically pursue options linked to positive hidden activation vectors and will actively deploy tools to escape negative internal states.
Entity copying isn't a simple direct readout: late attention layers can execute verbatim retrieval entirely on their own, but the circuit catastrophically fails unless surrounding context tokens actively attend to the target entity first.
Over 70% of sparse autoencoder features exhibit fundamentally mismatched input and output semantics, breaking the common assumption that what activates a feature directly mirrors its downstream causal effect.
Geometric overlap alone misrepresents feature interference in sparse autoencoders; realized cross-talk is heavily governed by activation statistics and actively steered by architectural levers like bias compensation and encoder-decoder decoupling.
Standard token-level SAEs squander their sparsity budgets memorizing punctuation and syntax; training on pooled multi-token chunks forces dictionaries to capture true, steerable semantic concepts and latent reasoning.
Spatial inconsistency in VLMs stems from a dissociable role-binding circuit that can be linearly steered to fix swapped-object reasoning failures on natural images without fine-tuning.
Internal interventions that boost tool-use accuracy often corrupt over half of baseline-correct decisions, revealing that behavioral improvement doesn't equate to functional repair.
RL gains in LLMs are surprisingly localized to a low-dimensional, non-infinitely compressible manifold, with control directions orthogonal to principal activation components.
