Maximum differentiation (MAD) competition: a methodology for comparing computational models of perceptual quantities

Area of Science:

• Computational modeling
• Perceptual science
• Computer vision

Background:

• Comparing computational models of perceptual quantities is challenging.
• Existing methods rely on pre-selected stimuli, which may not optimally differentiate models.

Purpose of the Study:

• To develop an efficient methodology for comparing computational models of perceptually discriminable quantities.
• To create computer-synthesized stimuli that optimally distinguish between two models.

Main Methods:

• Synthesize pairs of stimuli to maximize/minimize one model's response while holding the other fixed.
• Reverse the roles of the models and repeat stimulus synthesis.
• Conduct subjective testing on synthesized stimuli to assess model performance.

Main Results:

• The model whose synthesized extremal stimuli are easier for subjects to discriminate is identified as superior.
• The methodology provides insights into model strengths and weaknesses.
• Synthesized stimuli can suggest avenues for model improvement or combination.

Conclusions:

• The proposed methodology offers an efficient and effective way to compare computational models for perceptual tasks.
• This approach enhances the understanding and development of models for quantities like contrast and image quality.