Search papers, labs, and topics across Lattice.
This paper introduces ArtAnno, an innovative artwork annotation system that leverages Bidirectional Human-AI Augmentation (BiHAA) to enhance the efficiency and effectiveness of annotating implicit semantics in art. By integrating real-time interactions between human annotators and AI agents, the system not only provides proactive support through semantic mining but also evolves its capabilities through continuous feedback from users. Evaluations show that ArtAnno significantly improves annotation efficiency and reduces the cognitive load on annotators, particularly those with limited domain knowledge.
Real-time collaboration between AI and human annotators can transform the efficiency of artwork annotation, reducing cognitive load and enhancing knowledge accumulation.
High-quality annotation of artworks is essential for computational art research, yet extracting implicit semantics remains challenging due to the reliance on culturally grounded meanings and deep contextual knowledge behind the images. Current AI-assisted annotation tools often lack assistance or rely on one-way workflows where experts have to perform extra manual calibrations to improve AI models, resulting in limited efficiency. To address this, we propose Bidirectional Human-AI Augmentation(BiHAA), a closed-loop framework in which skills and domain knowledge base evolve through real-time interaction and bidirectional HAI augmentation. Informed by a formative study with 20 artwork annotators from different backgrounds, we implement this framework in ArtAnno, an artwork annotation system driven by a multi-agent architecture. The system includes a Proactive Agentic Support Module, where AI augments humans through semantic mining and label suggestion, and an Interaction-Driven Evolution Module, where human expertise continuously enhances the AI through distilling annotation trajectories into reusable experience. Evaluation through a user study and two case studies demonstrates that our framework and system improve annotation efficiency, enable knowledge accumulation, and reduce the effort of information seeking and verification for annotators with limited domain expertise. We conclude by discussing broader implications and future directions.
