Generalization Effects of k-Neighbor Interpolation Training

Area of Science:

• Machine Learning
• Artificial Intelligence
• Neural Networks

Background:

• Conventional neural network training uses point-to-point mapping, which lacks accuracy between training points.
• This limitation can lead to overfitting and reduced performance in pattern classification tasks.

Purpose of the Study:

• To introduce a new training method, k-neighbor interpolation training (KNIT), for continuous mapping and pattern classification neural networks.
• To address the limitations of point-to-point mapping by incorporating local sample-density smoothing.

Main Methods:

• The study proposes interpolation training, a theory for line-to-line mapping.
• This is expanded to the k-neighbor interpolation training (KNIT) method, which maps line segments between k-nearest neighbors in the input space to the output space.
• KNIT creates a web structure of input samples mapped to a similar structure in the output space, interpolating output values.

Main Results:

• The KNIT method effectively reduces the overlearning problem associated with point-to-point training by smoothing input/output functions.
• Simulations demonstrated that KNIT significantly improves vowel recognition accuracy on a small speech database.

Conclusions:

• KNIT offers a more robust and accurate training approach for neural networks compared to traditional point-to-point methods.
• The findings suggest KNIT's potential for enhancing performance in various pattern classification and continuous mapping applications.