Deep Residual Learning for Nonlinear Regression.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Deep learning has significantly advanced machine learning capabilities.
• Accurate regression of nonlinear functions is crucial in various scientific and engineering fields.
• Existing methods may have limitations in capturing complex nonlinear relationships.

Purpose of the Study:

• To develop and evaluate a deep residual neural network (ResNet) for nonlinear function regression.
• To identify optimal parameters (depth and width) for the proposed ResNet model.
• To compare the performance of the ResNet regression model against other approximation techniques.

Main Methods:

• A deep residual neural network architecture was designed, replacing convolutional and pooling layers with fully connected layers within residual blocks.
• Neural networks of varying depths and widths were trained and tested on simulated data to determine optimal parameters.
• The optimal ResNet model underwent extensive numerical testing on simulated data.
• Performance was benchmarked against linear and nonlinear methods including lasso regression, decision trees, and support vector machines.

Main Results:

• The developed ResNet model demonstrated robust performance on simulated data.
• The optimal ResNet regression model exhibited superior approximation capacity compared to lasso regression, decision trees, and support vector machines.
• The model showed stability and practical applicability when applied to real-world relative humidity series prediction.

Conclusions:

• The proposed deep residual neural network offers an effective approach for nonlinear function regression.
• The ResNet model provides a more accurate and capable alternative to existing regression techniques.
• The model's stability and applicability are confirmed through simulations and a real-world case study.