Related Experiment Videos
Spurious valleys in the error surface of recurrent networks--analysis and avoidance
Jason Horn1, Orlando De Jesús, Martin T Hagan
1Agilent Technologies High Frequency Technology Center, Santa Clara, CA 95051 USA. jason@jasonhorn.com
Abstract:
This paper gives a detailed analysis of the error surfaces of certain recurrent networks and explains some difficulties encountered in training recurrent networks. We show that these error surfaces contain many spurious valleys, and we analyze the mechanisms that cause the valleys to appear. We demonstrate that the principle mechanism can be understood through the analysis of the roots of random polynomials. This paper also provides suggestions for improvements in batch training procedures that can help avoid the difficulties caused by spurious valleys, thereby improving training speed and reliability.
Related Concept Videos
Avoidance Learning and Learned Helplessness
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Propagation of Uncertainty from Random Error
Propagation of Uncertainty from Systematic Error
Types of Errors: Detection and Minimization
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Detection of Gross Error: The Q Test
Uniform Depth Channel Flow: Problem Solving