Hilbert matrix-based weight initialization enhanced by mutual information for neural network optimization.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Science

Background:

• Artificial Neural Networks (ANNs) are widely used for approximation and regression tasks.
• ANN training efficiency is significantly influenced by weight initialization strategies.
• Existing methods may not fully optimize convergence speed and learning stability.

Purpose of the Study:

• To introduce an innovative weight initialization technique for ANNs.
• To accelerate the training convergence of ANNs.
• To improve the learning stability of ANN models.

Main Methods:

• Developed a novel weight initialization system combining Mutual Information (MI) for feature selection and the Hilbert matrix method.
• Ranked features using MI scores and distributed them across a scaled Hilbert matrix.
• Assigned weights based on feature rank to prioritize higher-ranked elements.

Main Results:

• The proposed MI-Hilbert weight initialization approach demonstrated superior performance across multiple datasets.
• Achieved faster training convergence compared to conventional methods.
• Maintained robust learning stability, validated by Mean Squared Error (MSE) and R2 metrics.

Conclusions:

• The integration of MI-based feature ranking and Hilbert-matrix-based weight initialization offers a significant advancement in ANN training.
• This novel technique enhances both the speed of convergence and the stability of the learning process.
• The MI-Hilbert method presents a promising solution for optimizing ANN performance in various applications.