Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces VisTCP, a visualization framework designed to enhance the semantic understanding of Traditional Chinese Paintings (TCPs) by integrating expert knowledge with an intelligent model for structured representation. The framework addresses significant challenges in accurately identifying ancient objects and events in TCPs, which often lead to semantic misunderstandings when using conventional image-oriented methods. Through expert-annotated data and a human-in-the-loop approach, VisTCP enables art historians to iteratively refine structured representations, demonstrating its effectiveness in a case study with real datasets.
Art historians can now achieve reliable structured representations of Traditional Chinese Paintings by leveraging a novel framework that combines expert insights with intelligent modeling.
Structured representation can characterize semantic objects and relationships in images. It provides a possible effective way for the semantic understanding of Traditional Chinese Paintings (TCPs) to better support archaeology and art history research. However, most image-oriented structured representation methods perform poorly on TCPs, due to two major challenges: 1) the objects and events of TCPs exhibit substantial differences from modern natural images, which results in semantic misunderstandings of TCPs; and 2) it is difficult to achieve accurate identification of ancient objects and events in TCPs, even for domain experts.In this paper, we propose VisTCP, a visualization framework that combines a TCP-oriented intelligent model and expert knowledge, which enables art historians to achieve trustworthy structured representations of TCPs in a human-in-the-loop manner. Firstly, we conduct a pilot study with three domain experts to build a semantic taxonomy of TCPs. Then, expert-annotated data are used to train a TCP-oriented structured representation model, which can automatically extract meaningful objects and their relationships in TCPs. To inform users of the model uncertainty, we design a joint embedding visualization view to show the differences between expert annotations and model predictions. This allows users to refine the structured representation based on their domain knowledge, enabling iterative optimization of the model. Finally, we conduct a case study, a usage scenario, and expert interviews on a real dataset to demonstrate the effectiveness of VisTCP in supporting the structured representation and semantic understanding of TCPs.