Search papers, labs, and topics across Lattice.
This paper introduces TextRefine, a post-training framework designed to enhance text editing in product posters by addressing issues of textual fidelity, spatial placement, and glyph rendering. By employing a dual reward system that evaluates both semantic fidelity and spatial conflicts, TextRefine effectively improves the accuracy of text insertion and replacement tasks, outperforming existing image editing models. The introduction of the OpenTextEdit dataset, featuring 100K images specifically tailored for this task, further supports the framework's robust evaluation and demonstrates significant advancements in maintaining source-image integrity while editing text.
TextRefine achieves superior text editing in product posters by ensuring high fidelity and optimal placement, overcoming common pitfalls of existing models.
Text editing in product posters entails inserting new text or replacing existing text while preserving product appearance, background content, and global composition. Despite recent progress in instruction-based image editing, general-purpose models remain unreliable in this setting: they often omit or incorrectly render the target text, place it over salient products or pre-existing content, and produce structurally distorted or visually inconsistent glyphs. We introduce \textbf{TextRefine}, a task-aligned post-training framework that combines supervised fine-tuning with operation-specific reward optimization to address these complementary failure modes. For text insertion, our text-span-level reward jointly assesses semantic fidelity and target-span coverage, penalizes spatial conflicts with products and existing text, and employs a gated structural constraint to preserve non-text regions. For text replacement, our glyph-level reward leverages the connectionist temporal classification (CTC) posterior of the target character to provide graded supervision for fine-grained defects, including missing strokes, structural deformations, and confusion among visually similar characters. We further introduce \textbf{OpenTextEdit}, a dataset comprising 100K images for text editing in product posters, with multi-text layouts, detailed text attributes, product masks, and challenging low-frequency characters. Extensive experiments on both insertion and replacement demonstrate that TextRefine consistently outperforms the evaluated image editing baselines in textual fidelity, placement reliability, and glyph quality while better preserving source-image content.
