Write a Classifier: Predicting Visual Classifiers from Unstructured Text.

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Humans learn visual concepts through text and speech descriptions.
• Machine learning often requires numerous labeled images for training visual classifiers.

Purpose of the Study:

• To computationally model the learning process of visual classifiers using only textual descriptions.
• To develop methods for learning explicit visual classifiers without any training images.

Main Methods:

• Investigated baseline formulations based on regression and domain transfer for predicting linear classifiers.
• Proposed a constrained optimization formulation combining regression and knowledge transfer.
• Developed generic kernelized models utilizing Reproducing Kernel Hilbert Space (RKHS) kernels.
• Introduced a novel kernel function for unstructured text based on distributional semantics.

Main Results:

• The proposed constrained optimization and kernelized models successfully predicted visual classifiers.
• The final model demonstrated superior performance over designed baselines on challenging datasets.
• The distributional semantics-based kernel function showed advantages in this specific application.

Conclusions:

• It is feasible to learn explicit visual classifiers solely from textual descriptions.
• The developed kernelized approach offers a powerful framework for text-to-visual learning.
• This research opens avenues for zero-shot visual learning and knowledge transfer.