Search papers, labs, and topics across Lattice.
The paper introduces Imagine, a zero-shot commonsense reasoning framework that integrates visual knowledge by embedding an image generator into the reasoning pipeline of pre-trained language models (PLMs). To effectively utilize the imagined visual context, the authors construct synthetic datasets emulating visual question-answering scenarios. Experiments on commonsense reasoning benchmarks show that Imagine outperforms existing zero-shot methods and even surpasses advanced large language models, demonstrating the ability of machine imagination to mitigate reporting bias and enhance generalization.
Machine-generated images can significantly boost zero-shot commonsense reasoning in language models, outperforming even larger models by mitigating biases in textual knowledge.
Recent advancements in zero-shot commonsense reasoning have empowered Pre-trained Language Models (PLMs) to acquire extensive commonsense knowledge without requiring task-specific fine-tuning. Despite this progress, these models frequently suffer from limitations caused by human reporting biases inherent in textual knowledge, leading to understanding discrepancies between machines and humans. To bridge this gap, we introduce an additional modality to enrich the reasoning capabilities of PLMs. We propose Imagine (Machine Imagination-based Reasoning), a novel zero-shot commonsense reasoning framework that supplements textual inputs with visual signals from machine-generated images. Specifically, we enhance PLMs with the ability to imagine by embedding an image generator directly into the reasoning pipeline. To facilitate effective utilization of this imagined visual context, we construct synthetic datasets designed to emulate visual question-answering scenarios. Through comprehensive evaluations on multiple commonsense reasoning benchmarks, we demonstrate that Imagine substantially outperforms existing zero-shot approaches and even surpasses advanced large language models. These results underscore the capability of machine imagination to mitigate reporting bias and significantly enhance the generalization ability of commonsense reasoning models
