Search papers, labs, and topics across Lattice.
This paper investigates the privacy vulnerabilities of domain-specific multimodal large language models (MLLMs) in Key Information Extraction (KIE) tasks, particularly when processing identity documents with insufficient visual evidence. It reveals that these models can leak sensitive personal information by relying on memorized field relations from training data, especially in the absence of visual cues. The authors introduce the Dynamic Relational Unlearning Framework (DRUF), which effectively reduces privacy leakage by 4.8 percentage points while preserving KIE performance, and present DocPrivacyBench to evaluate privacy risks systematically.
MLLMs can leak sensitive personal information when visual evidence is lacking, but the Dynamic Relational Unlearning Framework (DRUF) significantly mitigates this risk without sacrificing performance.
While the privacy risks of multimodal large language models (MLLMs) have drawn significant attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on document understanding MLLMs for identity document processing, this paper investigates the privacy issues inherent in Key Information Extraction (KIE) tasks. We reveal that when input images lack sufficient visual evidence, these models often rely on memorized field relations from training data to infer missing content, thereby leaking multiple correlated fields containing sensitive personal information. To mitigate this risk, we make three key contributions.First, we propose the Dynamic Relational Unlearning Framework (DRUF) which comprises a Relational Decoupling Unlearning (RDU) module and a dynamic set update mechanism. It suppresses the leakage of high-risk field pairs while preserving KIE performance.Second, we introduce DocPrivacyBench, a novel benchmark to systematically evaluate a model's susceptibility to privacy leakage under conditions of absent or minimal visual evidence.Third, we evaluate three MLLMs and six unlearning methods using this benchmark, assessing both post-unlearning leakage suppression and utility preservation.Our results demonstrate that existing MLLMs consistently exhibit privacy leakage when visual evidence is scarce, particularly on noisier datasets. In contrast, DRUF outperforms the strongest baseline by improving leakage suppression by 4.8 percentage points, effectively mitigating privacy risks while maintaining robust document information extraction performance.
