Trainable fusion rules. I. Large sample size case

Area of Science:

• Machine Learning
• Artificial Intelligence
• Pattern Recognition

Background:

• Neural network ensembles leverage multiple base classifiers for improved performance.
• Designing effective fusion rules is critical for ensemble accuracy.
• Finite sample effects can significantly impact ensemble design and performance.

Purpose of the Study:

• To analyze finite sample effects in neural network ensembles.
• To compare continuous versus categorical outputs from base classifiers.
• To evaluate fixed versus trainable fusion rules, including weighted averaging.

Main Methods:

• Review of statistical pattern classification algorithms as trainable fusion rules.
• Analysis of base classifier output types (continuous vs. categorical).
• Comparison of fixed versus trainable fusion rules (e.g., simple average, weighted average).

Main Results:

• Continuous outputs may be less preferable than crisp outputs if expert networks specialize.
• Trainable fusion rules, particularly weighted averaging, can outperform simple averaging.
• Rarely employed linear and non-linear statistical rules show potential for neural network ensembles.

Conclusions:

• The choice between continuous and categorical outputs depends on expert network specialization.
• Trainable fusion rules offer advantages over fixed rules in certain scenarios.
• Further investigation into less common statistical rules for ensembles is warranted.