New results on error correcting output codes of kernel machines

Area of Science:

• Machine Learning
• Computer Science
• Statistical Learning

Background:

• Multiclass classification is a fundamental problem in machine learning.
• Error Correcting Output Codes (ECOC) offer a framework for multiclass classification using binary classifiers.
• Existing ECOC methods face challenges in decoding and model selection.

Purpose of the Study:

• To introduce a new decoding function for ECOC that improves the mapping of classifier outputs to class codewords.
• To address the open problem of model selection in ECOC by developing theoretical bounds for kernel machines.
• To enhance the performance and hyperparameter tuning of ECOC-based multiclass classification systems.

Main Methods:

• Developed a novel decoding function that combines classifier margins using estimated class conditional probabilities.
• Derived new theoretical results bounding the leave-one-out (LOO) error for ECOC of kernel machines.
• Conducted empirical evaluations using Support Vector Machines (SVMs) as base binary classifiers.

Main Results:

• The proposed decoding function demonstrated advantages over commonly used margin-based decoding functions.
• Empirical evaluations confirmed that the derived LOO error bounds effectively estimate kernel parameters for model selection.
• The new methods enhance the accuracy and tunability of ECOC for multiclass classification tasks.

Conclusions:

• The novel decoding function offers a significant improvement for ECOC-based multiclass classification.
• The theoretical bounds provide a robust method for model selection and hyperparameter tuning in kernelized ECOC.
• This work advances the practical application of ECOC by addressing key challenges in decoding and model selection.