Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
The authors reframe visual token compression in vision-language models through a parameterization lens, disentangling subspace basis truncation (compressibility) from coordinate organization (learnability) to overcome the grounding failures of pruning and the heavy overhead of learned resamplers. They instantiate this theory via Braco, a lightweight coder combining transform-basis truncation, budget-dependent orthogonal re-parameterization, and spatial residual pooling. Under extreme $23\times$--$144\times$ compression, Braco preserves up to 95.2% accuracy while slashing prefill FLOPs by over 84% and running with $78.8\times$ fewer compressor FLOPs than existing resamplers.
Vision-language models can retain over 95% of their task accuracy under extreme 144× token compression without relying on destructive token pruning or expensive cross-attention resamplers.
Visual-token compression is effective for improving the efficiency of vision-language models, but under extreme compression budgets, token pruning can break visual grounding while learned resamplers increase parameter count, attention cost, and training complexity. We revisit compression through a token parameterization lens, separating (i) basis transformation and structured truncation (retained subspace/compressibility) from (ii) coordinate organization (optimization and cross-modal alignment). This view yields two coupled objectives, compressibility and learnability, which we formalize as unified functionals. Guided by these objectives, we design Braco, a lightweight four-step coder that combines transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling. Experiments show that Braco forms the favorable empirical accuracy-efficiency frontier under $23\times$--$64\times$ compression and remains competitive at $144\times$, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%--86.7% relative to the uncompressed upper bound. Against prior methods, Braco matches or improves accuracy while achieving up to approximately 36% end-to-end speedup and using $16.6\times$/$78.8\times$ lower compressor latency/FLOPs.