Search papers, labs, and topics across Lattice.
This paper introduces MythraGen, a two-stage retrieval augmented generation framework that enhances text-to-artistic image generation by integrating an art retrieval mechanism with LoRA-based model fine-tuning. By leveraging a large-scale art dataset, MythraGen retrieves images with high similarity to the input prompts, which are then used to fine-tune the Stable Diffusion model, resulting in improved contextual accuracy and artistic fidelity. Experimental results demonstrate that MythraGen significantly outperforms existing methods in generating artworks that align closely with user inputs, as validated by user studies on the WikiArt dataset.
MythraGen achieves a new benchmark in text-to-art generation by combining retrieval techniques with model fine-tuning, producing artworks that resonate deeply with user prompts.
Text-to-image generation has seen rapid advancements, especially with the development of generative models. However, challenges remain in achieving high-quality, contextually accurate image outputs that faithfully match the provided textual descriptions, especially in artistic generation. In this paper, we present a simple yet efficient retrieval augmented generation framework, namely MythraGen, for text-to-artistic image generation by integrating an art retrieval mechanism with LoRA-based model fine-tuning. Our method extracts features from a large-scale art dataset, optimizing the generation process by combining artist-specific styles and content. Particularly, retrieved images from an external art database that have the highest similarity to the query prompt are used to finetune Stable Diffusion using LoRA for desired art generation. Experimental results and user studies on the WikiArt dataset show that our proposed method can generate artworks that closely match the user's input, significantly outperforming existing solutions.
