Search papers, labs, and topics across Lattice.
This paper introduces HyperVis, a novel approach that enhances compositional reasoning in Vision-Language Models (VLMs) by utilizing continuous latent visual relational graphs on the Lorentz hyperboloid, circumventing the limitations of discrete scene graph triplets. By computing a dense visual relation tensor through spatially-biased cross-attention and enforcing a hierarchical structure via spatial physics, HyperVis improves generative visual question answering (GQA) accuracy from 58.86% to 61.03% and boosts discriminative scoring in compositional tasks by 6.25 percentage points. The findings highlight that the learned curvature stabilizes at 魏=4.0, demonstrating the necessity of strongly curved space for effective representation of continuous visual features.
HyperVis reveals that leveraging hyperbolic geometry can significantly enhance compositional reasoning in VLMs, outperforming traditional discrete approaches.
Vision-Language Models (VLMs) struggle with compositional reasoning that requires understanding inter-object relationships. A natural remedy is to inject explicit scene graph triplets $\langle s, p, o \rangle$ from an off-the-shelf scene graph generator (SGG), but we show this backfires: discrete text labels collide with the continuous visual modality, degrading GQA accuracy from 60.38\% to 58.86\%. We propose \textbf{HyperVis}, which bypasses the SGG semantic bottleneck entirely. From $N$ class-agnostic region proposals, we compute a dense $O(N^2)$ visual relation tensor via spatially-biased cross-attention, project it onto a Lorentz hyperboloid, and enforce hierarchy through spatial physics, namely IoA-driven entailment cones and exterior-angle repulsion. We discover that HyperVis contributes in two complementary ways: (1) as a \emph{training-time regularizer}, the hyperbolic relational losses shape LoRA representations that improve generative VQA (GQA 61.03\% vs.\ 57.21\% for LoRA fine-tuning without relational losses, recovering and surpassing the baseline); and (2) as an \emph{inference-time relational encoder}, hyperbolic prefix tokens boost discriminative compositional scoring (SugarCrepe 79.94\%, $+$6.25pp over baseline). The learned curvature stabilises at $魏{=}4.0$, an order of magnitude above prior hyperbolic VLMs where $魏$ typically collapses toward zero, indicating that continuous visual features genuinely require the exponential volume of strongly curved space. A controlled Euclidean ablation confirms this decomposition: the relational pipeline regularises LoRA comparably in flat space (GQA 60.81\%), but the compositionality gain is specifically hyperbolic (SugarCrepe $+$4.58pp over Euclidean), with entailment loss ${\sim}6{\times}$ higher in Euclidean training. Codes are available at TBA.
