Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces ReasonEdit, a novel model editing framework for vision-language models (VLMs) specifically designed to address reasoning-heavy tasks. ReasonEdit incorporates human reasoning by storing explanations in a codebook and retrieving relevant facts during inference using a topology-balanced multimodal embedding method. Experiments across four VLMs and multiple rationale-based VQA datasets demonstrate that ReasonEdit achieves state-of-the-art editing performance and improves edit generalization by leveraging human reasoning.
Injecting human-provided reasoning into VLM editing boosts generalization, outperforming existing methods on reasoning-heavy visual tasks.
Model editing aims to correct errors in large, pretrained models without altering unrelated behaviors. While some recent works have edited vision-language models (VLMs), no existing editors tackle reasoning-heavy tasks, which typically require humans and models to reason about images. We therefore propose ReasonEdit, the first VLM editor to let users explain their reasoning during editing, introducing a new, practical model editing setup. ReasonEdit continuously stores human reasoning in a codebook, and retrieves only relevant facts during inference using a novel topology-balanced multimodal embedding method inspired by network science. Across four VLMs on multiple rationale-based visual question answering datasets, ReasonEdit achieves state-of-the-art editing performance, ultimately showing that using human reasoning during editing greatly improves edit generalization.