Building a Chinese ancient architecture multimodal dataset combining image, annotation and style-model

Area of Science:

• Computer Vision and Artificial Intelligence
• Digital Humanities and Cultural Heritage

Background:

• Diffusion models excel in multimodal generation but struggle with specialized domains like Chinese ancient architecture due to data scarcity.
• Existing datasets lack comprehensive visual and textual information specific to unique Chinese architectural features.

Purpose of the Study:

• To develop an extensive multimodal dataset for Chinese ancient architecture, spanning the Tang to Yuan Dynasties.
• To overcome data limitations and improve AI's ability to generate and understand Chinese ancient architectural imagery.

Main Methods:

• Construction of a multimodal dataset including images, videos, and style models.
• Categorization of visual data by location and annotation of images at two levels (initial and descriptive).
• Inclusion of seven artistic style fine-tuning models and Pinyin annotation for unique architectural terms.

Main Results:

• Creation of the first comprehensive multimodal dataset specifically for Chinese ancient architecture.
• The dataset provides rich, categorized, and annotated data crucial for training AI models.
• Pioneering use of Pinyin for annotating specialized Chinese architectural terminology.

Conclusions:

• The developed dataset significantly advances multimodal AI capabilities in depicting Chinese ancient architecture.
• This resource facilitates further research and innovation in AI-driven cultural heritage applications.
• The dataset serves as a foundational resource for preserving and disseminating knowledge of Chinese architectural history.