Using CNN Features to Better Understand What Makes Visual Artworks Special

Area of Science:

• Computational aesthetics
• Computer vision
• Neuroaesthetics

Background:

• Understanding the unique properties of visual artworks is a key goal in computational aesthetics.
• Computer vision techniques can analyze image statistics to differentiate artworks from non-art images.
• This research aims to inform theories about the neural mechanisms of aesthetic perception.

Purpose of the Study:

• To define variance measures of Convolutional Neural Network (CNN) features to distinguish artworks from non-art images.
• To investigate the properties of feature responses in artworks related to 'richness' and 'variability'.

Main Methods:

• Utilized a well-established CNN trained on millions of images.
• Analyzed an image dataset comprising traditional Western, Islamic, and Chinese art, alongside non-art images.
• Defined and applied two variance measures to capture CNN feature response patterns.

Main Results:

• Achieved a high classification accuracy of 93.0% in distinguishing artworks from non-art images using two variance measures.
• The first measure ('richness') indicates diverse CNN features respond to pictorial elements within artwork subregions.
• The second measure ('variability') shows a high diversity in individual CNN feature responses across artwork subregions.

Conclusions:

• A combination of 'richness' and 'variability' in CNN feature responses may define unique properties of traditional visual artworks.
• These findings suggest potential neural underpinnings for the perceptual quality of art.
• Future research should explore these qualities in other aesthetic domains like music and literature.