Search papers, labs, and topics across Lattice.
This paper introduces a model-agnostic framework for refining Scene Graph Generation (SGG) predictions by leveraging visual commonsense knowledge, addressing the limitations of existing models under annotation sparsity. By systematically mining commonsense-grounded constraints and applying declarative reasoning during inference, the approach enhances the reliability of SGG outputs without the need for manual rule crafting or model retraining. The proposed method shows significant performance improvements across three standard benchmarks, highlighting the effectiveness of integrating structured reasoning into SGG tasks.
By integrating commonsense reasoning into scene graph generation, this framework achieves consistent performance boosts, even in sparsely annotated scenarios.
Learning-driven Scene Graph Generation (SGG) models excel on frequent relation types but degrade sharply under annotation sparsity, failing to capture reliable visual commonsense knowledge. We propose a model-agnostic, semantically-guided knowledge refinement framework that systematically mines commonsense-grounded constraints from training data - capturing spatial, functional, and qualitative relational regularities - and uses general declarative commonsense reasoning to correct and refine ranked SGG predictions at inference time. The framework requires no manual rule authoring, no model retraining, and transfers across datasets and architectures. On three standard benchmarks, we obtain consistent improvements over strong baselines, demonstrating that structured visual commonsense reasoning over deep scene semantics is a practical and effective complement to purely learning-based scene graph generation.
