Search papers, labs, and topics across Lattice.
This paper introduces CARVE, a training-free framework designed to optimize the allocation of visual tokens in slice-based multi-layered language models (MLLMs) for 3D medical volume understanding. By analyzing the diminishing returns of increasing the visual token budget, the authors demonstrate that selectively reallocating tokens based on cross-slice evidence significantly enhances performance while reducing redundancy. CARVE achieves an impressive 80% reduction in visual tokens with a retention of 98.1% performance across multiple benchmarks, outperforming existing compression methods.
CARVE slashes visual token usage by 80% while maintaining nearly all performance, reshaping how we approach token allocation in 3D medical imaging.
Slice-based MLLMs leverage mature 2D encoders by representing 3D volumes as sequences of 2D slices. However, this slice-wise formulation produces thousands of visual tokens that burden the LLM backbone, many of which capture overlapping visual evidence across adjacent slices. To understand how effectively a growing visual token budget improves performance, we perform scaling analyses on two 3D medical VQA benchmarks and find diminishing returns: cost keeps rising while accuracy saturates, and improving in-plane resolution is more effective than adding slices at comparable budgets. The budget should therefore be allocated more selectively rather than simply enlarged, yet most token compression methods are designed for 2D images or videos, where redundancy arises from spatial layout or temporal motion rather than from near-duplicate content along the depth axis. We present CARVE, a training-free framework that compresses visual tokens prior to LLM inference and casts token reduction as budget-constrained 2.5D allocation. CARVE partitions the depth axis into coherent windows and allocates tokens non-uniformly according to normalized cross-slice evidence. Under a shared budget, CARVE builds spatial anchors on representative slices and retrieves locally varying evidence from the full volume, then merges remaining eligible tokens into nearby anchors within each window. Removing roughly 80% of the visual tokens on Hulu-Med-7B, CARVE leads all compression baselines on every AMOS-MM report-generation metric, with 6.2 points higher retention of full-token quality than the strongest baseline, and preserves 98.1% of full-token performance across three VQA benchmarks.
