A theoretical and experimental analysis of linear combiners for multiple classifier systems.

Area of Science:

• Machine Learning
• Pattern Recognition
• Artificial Intelligence

Background:

• Linear combiners are widely used in multiple classifier systems for pattern classification.
• Theoretical understanding of linear combiner operation, particularly regarding optimal weighting, is limited.
• Existing frameworks, like Tumer and Ghosh's, provide a basis for analyzing combiner performance.

Purpose of the Study:

• To provide a theoretical and experimental analysis of linear combiners in multiple classifier systems.
• To investigate the impact of individual classifier performance and output correlation on misclassification probability.
• To evaluate the performance gains of weighted averaging over simple averaging for optimal weights.

Main Methods:

• Theoretical analysis of linear combiner performance based on classifier outputs and correlations.
• Focus on non-negative weights for individual classifiers, considering ideal (optimal) weight scenarios.
• Experimental validation using real datasets to compare model predictions with observed behavior.

Main Results:

• Theoretical model demonstrates the dependence of misclassification probability on individual classifier performance and output correlation.
• Quantified ideal performance improvement achievable with weighted averaging compared to simple averaging.
• Experimental results confirm the agreement between the analytical model and real-world data.

Conclusions:

• The theoretical model accurately predicts the behavior of linear combiners in multiple classifier systems.
• Optimal weighting in linear combiners can significantly enhance performance over simple averaging.
• The study contributes theoretical insights and practical relevance to the field of multiple classifier systems.