Can machine learning account for human visual object shape similarity judgments?

Area of Science:

• Cognitive Science
• Computer Vision
• Machine Learning

Background:

• Human visual shape perception is complex and involves viewpoint invariance.
• Previous metric learning studies often used single-viewpoint object representations.
• Understanding how machine learning models replicate human similarity judgments is crucial.

Purpose of the Study:

• To analyze metric learning system performance on viewpoint-invariant shape similarity judgments.
• To compare different object representations for predicting human similarity data.
• To investigate deep neural network (DNN) limitations in learning viewpoint-invariant features.

Main Methods:

• Collected human similarity judgments for "Fribbles" (part-based objects) from multiple viewpoints.
• Trained and evaluated metric learning systems using pixel-based, DNN-based, and part-based representations.
• Analyzed DNN performance, particularly those trained with triplet loss functions.

Main Results:

• A viewpoint-invariant, part-based representation accurately predicted human shape similarity judgments.
• Pixel-based and standard DNN-based representations failed to explain the data.
• DNNs trained with triplet loss performed poorly, likely due to optimization nonconvexity.

Conclusions:

• Viewpoint insensitivity is essential for human visual shape perception.
• Current DNNs require specific training strategies to achieve viewpoint-invariant representations.
• Future machine learning models must learn viewpoint-insensitive features to model human similarity judgments effectively.