Trainable fusion rules. II. Small sample-size effects

Area of Science:

• Machine Learning
• Artificial Intelligence
• Pattern Recognition

Background:

• Neural network ensembles can improve classification but are sensitive to small sample sizes.
• Understanding small sample effects on fusion rules is crucial for reliable classifier design.

Purpose of the Study:

• To theoretically analyze small-sample properties of trainable fusion rules.
• To determine conditions under which neural network ensembles improve or degrade classification.
• To investigate the impact of small learning sets on fusion rule performance.

Main Methods:

• Theoretical analysis of trainable fusion rules (linear weighted average, Behavior-Knowledge-Space).
• Examination of small sample effects: training bias, self-boasting, and complexity determination.
• Study of correction terms for self-boasting and noise injection techniques.

Main Results:

• Small learning sets increase expert classifier error and damage output correlation.
• Non-trainable fusion rules can outperform trainable ones with insufficient training data.
• Noise injection reduces fusion rule complexity and expert boasting bias.

Conclusions:

• Small sample sizes pose significant challenges for trainable fusion rules in neural network ensembles.
• Careful consideration of training data size and fusion rule complexity is essential.
• Techniques like noise injection can improve ensemble performance under data limitations.