What makes Mona Lisa smile?

To study the ability of humans to read subtle changes in facial expression, we applied reverse correlation technique to reveal visual features that mediate understanding of emotion expressed by the face. Surprising findings were that (1) the noise added to a test face image had profound effect on the facial expression and (2) in almost every instance the new expression was meaningful. To quantify the effect, we asked naïve observers to rank the face of Mona Lisa superimposed with noise, based on their perception of her emotional state along the sad/happy dimension. Typically, a hundred trials (with 10 or more samples for each rank category) were sufficient to reveal areas altering the facial expression, which is about two orders of magnitude less than in the other reverse correlation studies. Moreover, the perception of smiling in the eyes was solely attributable to a configurational effect projecting from the mouth region.