Search papers, labs, and topics across Lattice.
This paper introduces Counterfactual Visual Evidence-Guided Selective Attribute Indexing (CVE-SAI), a novel method designed to enhance the accuracy of e-commerce product attribute indexing by integrating visual evidence verification. By employing Focus-Zone Distortion (FZD) and Evidence-Guided Attention Redistribution (EGAR), the approach effectively distinguishes between transient predictions and persistent index admissions, ensuring that only visually supported attributes are indexed. Experimental results demonstrate that CVE-SAI significantly improves attribute inference and retrieval performance while maintaining a low rate of unsafe admissions, thereby addressing critical gaps in current multimodal product models.
CVE-SAI achieves the highest certified admission coverage while minimizing unsafe auto-induced exposure, revolutionizing how e-commerce attributes are indexed and verified.
Multimodal product models can complete missing e-commerce attributes, yet current methods still optimize attribute-answer accuracy without verifying visual support, conflate transient prediction with persistent index admission, and lack explicit risk control over factually incorrect or visually unsupported values. We address these gaps with Counterfactual Visual Evidence-Guided Selective Attribute Indexing (CVE-SAI), which first infers and freezes an ontology-constrained candidate from the primary image and attribute question without catalog text, and then decides whether that candidate should enter the index. Focus-Zone Distortion (FZD) constructs an attribute-specific visual-dependence proxy through a controlled counterfactual intervention, and Evidence-Guided Attention Redistribution (EGAR) uses the proxy to refine ontology-constrained scoring. The canonical candidate is frozen before evidence necessity, evidence retention, nuisance-transformation stability, and candidate-specific catalog-text conflict audits; catalog text can only tighten admission and cannot revise the candidate. Independent family-level calibration selects one policy with a simultaneous one-sided finite-sample bound under a 5% unsafe-admission budget. Experiments on five visual attributes derived from Amazon Berkeley Objects show that CVE-SAI improves attribute inference and evidence localization, achieves the highest certified admission coverage under the shared risk protocol, and yields the strongest controlled retrieval performance with the lowest unsafe auto-induced exposure among automatic-admission systems. Separating inference from admission therefore enables visually supported attribute completion to improve retrieval while limiting persistent index contamination.
