Search papers, labs, and topics across Lattice.
This paper introduces the MathVis-Fine framework, which addresses the shortcomings of existing multimodal reasoning approaches by modeling fine-grained visual dependencies in mathematical problem-solving. By constructing a specialized dataset with detailed visual annotations and implementing a two-stage progressive training paradigm, the authors adapt supervisory signals to the necessity of visual information for each specific sample. The results show that this method significantly improves the accuracy of multimodal reasoning by balancing rewards based on intrinsic visual dependencies, thus enhancing visual perception in mathematical contexts.
Fine-grained visual dependencies can drastically improve multimodal reasoning accuracy in mathematical problem-solving, challenging the notion that visual inputs are merely auxiliary.
Chain-of-Thought (CoT) reasoning has extended from purely linguistic domains to multimodal scenarios; however, existing approaches often treat visual inputs as homogeneous or auxiliary signals, failing to capture the intricate and sample-specific dependencies between text and images in mathematical problem-solving. This gives rise to two core issues: first, the supervisory signals for visual content are generalized and coarse-grained, lacking adaptation to the actual necessity of visual information in each sample; second, training feedback becomes inaccurate when visual rewards are uniformly applied without distinguishing the complementary relationships among inputs. These limitations hinder models from achieving precise multimodal reasoning. In this work, we propose a framework for modeling fine-grained visual dependencies in mathematical reasoning. We first construct the MathVis-Fine dataset, augmenting fine-grained visual annotations with visual dependency ratings. Building upon this dataset, we introduce a two-stage progressive visual enhancement training paradigm that balances answer correctness rewards and visual grounding rewards according to the intrinsic visual dependency level of each sample, thereby mitigating reward bias and improving supervision accuracy. Extensive experiments demonstrate that the MathVis-Fine framework effectively enhances visual perception progressively based on visual dependency, offering a more precise training framework for multimodal mathematical reasoning. We will release the dataset upon acceptance.
