相关实验视频
Updated: May 24, 2025

13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
8.9K
在规范化的实证风险最小化中使用稀疏深度RELU网络进行近乎最佳的学习,使用Lipschitz损失最小化
Ke Huang1, Mingming Liu2, Shujie Ma3
1Department of Statistics, University of California, Riverside, Riverside 92521, CA, U.S.A. khuan049@ucr.edu.
Neural computation
|March 3, 2025
概括
我们为回归问题引入了稀疏深度ReLU网络 (SDRN). 这种新型估计器实现了接近最佳的融合率,通过减少参数的过度拟合来减轻传统网络的性能.
科学领域:
- 机器学习 机器学习
- 统计学学习理论
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 经验风险最小化是统计估计的标准方法.
- 利普希茨损失函数通常用于回归和分类.
- 深度神经网络经常在过拟合和参数效率方面扎.
研究的目的:
- 为回归函数提出一个稀疏深度RELU网络 (SDRN) 估计器.
- 为 SDRN 估计器建立非对称的超额风险边界.
- 分析SDRN的收率和参数复杂性.
主要方法:
- 开发一个稀疏深度ReLU网络架构.
- 使用正规化的经验风险最小化与利普希茨损失函数.
- 导出Sobolev空间与混合衍生品的非对称过度风险极限.
主要成果:
- 该SDRN估计器实现了近乎最佳的最小合率,与1D非参数回归可比.
- 当定位时,收率在特征维度上是对数的,当维度随样本大小增长时略慢一些.
- 随着样本大小的增加,SDRN的深度以对数方式增长,而节点/重量则以多项式方式增长.
结论:
- 拟议的SDRN估计器有效地估计了回归函数.
- 在传统的前网络中,SDRN克服了过问题.
- SDRN提供了更好的深度和参数效率,以提高回归性能.
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
38
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
38
Residuals and Least-Squares Property
7.2K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.2K
Residual Plots
4.5K
A residual plot is a statistical representation of data used to analyze correlation and regression results. It helps verify the requirements for drawing specific conclusions about correlation and regression. To obtain the residual plot, first, the residual for each data value is calculated, which is simply the vertical distance between the observed and the predicted value obtained from the regression equation.
When the residual values are plotted against the variable x, it is called a residual...
When the residual values are plotted against the variable x, it is called a residual...
4.5K
Empirical Method to Interpret Standard Deviation
5.1K
The empirical rule, also known as the three-sigma rule, allows a statistician to interpret the standard deviation in a normally distributed dataset. The rule states that 68% of the data lies within one standard deviation from the mean, 95% lies within two standard deviations from the mean, and 99.7% lies within three standard deviations from the mean. Additionally, this rule is also called the 68-95-99.7 rule.
This rule is used widely in statistics to calculate the proportion of data values...
This rule is used widely in statistics to calculate the proportion of data values...
5.1K
Purposive Learning
96
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
96

