Related Experiment Video
Updated: Jul 1, 2025
13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
9.2K
Learning Rates of Deep Nets for Geometrically Strongly Mixing Sequence
IEEE Transactions on Neural Networks and Learning Systems
|March 11, 2024
Summary
This study establishes a fast learning rate for deep neural networks (DNNs) using empirical risk minimization (ERM) with dependent samples. This work generalizes existing theories by addressing geometrically strongly mixing sequences, a first for DNN convergence research.
Area of Science:
- Machine Learning
- Theoretical Computer Science
- Statistics
Background:
- Deep learning's success necessitates a theoretical foundation.
- Existing deep neural network (DNN) convergence studies often assume independent samples, limiting real-world applicability.
Purpose of the Study:
- To establish a theoretical basis for deep neural network (DNN) convergence with dependent samples.
- To develop a fast learning rate for empirical risk minimization (ERM) in DNN regression under sample dependence.
Main Methods:
- Empirical risk minimization (ERM) for deep neural network (DNN) regression.
- Analysis of sample dependence using geometrically strongly mixing sequences.
- Establishing convergence rates for DNN training.
Main Results:
- A novel convergence result for DNNs utilizing mixing sequences, a first in the field.
- Demonstration of a fast learning rate for empirical risk minimization (ERM) with dependent samples.
- The findings generalize existing convergence results for independent samples.
Conclusions:
- This research provides the first convergence guarantee for DNNs with dependent samples based on mixing sequences.
- The established fast learning rate offers a more realistic theoretical framework for DNNs in practical applications.
- This work bridges a critical gap between theoretical assumptions and real-world data characteristics in deep learning.
