Search papers, labs, and topics across Lattice.
This paper introduces Visual Retrieval-Augmented Generation (Visual-RAG), a novel framework that generates animal art from natural silhouettes by retrieving structurally similar shapes from a large corpus of 28,586 silhouettes. The method employs diffusion-based generation techniques, specifically ControlNet and IP-Adapter, to enhance the fidelity of the generated images, with ablation studies indicating that structural alignment is crucial for achieving high-quality outputs. Despite promising results in silhouette fidelity and aesthetics, user evaluations highlight ongoing challenges in maximizing perceptual impact, suggesting that while machines can aid in imaginative processes, further refinement is needed.
Visual-RAG reveals that machines can mimic human-like creativity by transforming ambiguous silhouettes into compelling animal art, but still struggle with achieving high perceptual impact.
Generative AI has advanced the ability to render photorealistic or artistic images, yet it remains limited in a key aspect of human creativity: interpreting ambiguous shapes. This phenomenon, rooted in pareidolia, allows humans to perceive meaningful forms in random patterns such as clouds, stones, or leaves. To computationally replicate this imaginative process, we introduce Visual Retrieval-Augmented Generation (Visual-RAG), a framework that generates animal art directly from natural silhouettes. Our method retrieves structurally similar animal shapes from a curated corpus of 28,586 high-quality silhouettes and uses them as reference exemplars to guide diffusion-based generation with ControlNet and IP-Adapter. Ablation studies confirm that shape Context with RANSAC provides the most accurate alignment, while removing shape standardization reduces the inlier ratio to just 13.4\%, underscoring the importance of structural fidelity in Visual-RAG. A user study with 12 participants evaluated the outputs in terms of aesthetics, silhouette fidelity, and overall impression. Results reveal that while Visual-RAG provides plausible interpretations, challenges remain in achieving high perceptual impact. This work lays the foundation for computational pareidolia, showing how machines can contribute to the early stages of imaginative discovery.
