Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To resolve feature entanglement in dense vision-language representations, NeuronEye uses an overcomplete sparse concept basis to selectively activate query-relevant visual features and suppress dominant perceptual noise in a single forward pass. This top-down modulation enables frozen vision-language models to dynamically isolate and amplify weak but task-critical visual evidence guided by text prompts. Evaluated on Qwen2.5-VL-7B, the training-free framework yields consistent visual reasoning improvements, including +9.5 points on CV-Bench Distance and +8.3 points on BLINK Multi-view tasks.
Sparse neuron dictionaries are not just passive interpretability tools: actively steering frozen VLM tokens through query-guided concept activation delivers up to a +9.5 point accuracy boost on complex spatial reasoning without retraining.
Current vision-language models (VLMs) encode visual information in dense hidden states where object identity, spatial layout, and local attributes are implicitly entangled rather than explicitly disentangled, limiting their ability to isolate and modulate the specific visual evidence required by a given language query. Inspired by sparse population coding and top-down modulation in biological vision, we introduce NeuronEye, a plug-in framework that constructs a sparse, concept-level neuron vocabulary from intermediate VLM representations and selectively activates query-relevant visual concepts during inference. NeuronEye decomposes vision-token states into an overcomplete sparse basis organized by concept-level clusters, uses the language query to activate relevant clusters and localize the patches where selected concepts are expressed, and injects the focused evidence back into vision tokens. A complementary suppression mechanism attenuates dominant perceptual directions to preserve weaker but relevant cues. All operations run in a single forward pass over a frozen VLM backbone. On Qwen2.5-VL-7B, NeuronEye raises CV-Bench overall accuracy by +3.1 with gains of +9.5 on Distance, and improves BLINK Multi-view by +8.3, with similar trends on LLaVA-1.6-7B. These results suggest that sparse neuron vocabularies can serve not only as post-hoc interpretability tools but also as active interfaces for concept-level visual reasoning.