Related Experiment Videos
Context-aware multimodal AI navigates hidden pathways in five centuries of art evolution
Jin Kim1,2, Byunghwee Lee3,4, Taekho You5
1Department of Intelligent Semiconductors, Soongsil University, Seoul 06978, Republic of Korea.
Abstract:
The rise of multimodal generative AI transforms the intersection of technology and art, offering richer insights into large-scale artworks. While significant research has focused on their creative potential, their ability to represent artworks in latent spaces remains underexamined. We use generative AI, specifically Stable Diffusion, to analyze 500 y of Western paintings by extracting two types of latent information with the model: formal aspects (e.g., colors) and contextual aspects (e.g., subjects). Our findings reveal that contextual information exhibits stronger vector alignment and orientation with conventional artistic periods, styles, and individual artists than formal elements. Also, we show how artistic expression aligns with historical shifts using contextual keywords extracted from paintings. Our generative experiment, infusing prospective contexts into historical artworks, validates this vector alignment and orientation by synthesizing artworks consistent with the stylistic patterns of target periods. This study demonstrates how multimodal AI expands traditional formal analysis by integrating temporal, cultural, and historical contexts to quantify the latent structure of cultural knowledge.
Related Concept Videos
Sensory Modalities
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
Visual Agnosia
Parallel Processing
Gestalt Principles of Perception
Perception
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Natural and Artificial Concepts