Neural Basis of Perceptual Surface Qualities and Materials: Evidence from Electroencephalogram Decoding

The human visual system can easily recognize object material categories and estimate surface properties such as glossiness and smoothness. Recent psychophysical and computational studies suggest that the material perception depends on global feature statistics. To elucidate dynamic neural representations underlying surface property and material perception in humans, we measured visual evoked potentials (VEPs) for 191 natural images consisting of 20 categories of materials and then classified material categories and surface properties from the VEPs. As a result, we found that material categories were correctly classified by the VEPs even at latencies of 150 msec or less. The apparent surface properties were also significantly classified within 175 msec (lightness, colorfulness, and smoothness) and after 200 msec (glossiness, hardness, and heaviness). The subsequent reverse-correlation analysis revealed that the VEPs at these latencies are highly correlated with low- and high-level global feature statistics of the surface images, indicating that neural activities about such global features are reflected in the VEPs. Moreover, by using deep generative models (multimodal variational autoencoder models) to reconstruct surface images from the VEPs via style information, we demonstrated that the reconstructed surface images are judged by observers to have very similar material categories and surface properties as the original natural surfaces. These results support the notion that neural representations of statistical features in the early cortical response play a crucial role in the perception and recognition of surface materials in humans.