Related Experiment Videos
Linear independence of internal representations in multilayer perceptrons
1Harvard/MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
Abstract:
This investigation identifies the linear independence of the internal representation of the multilayer perceptron as an essential property for exact learning. The sigmoidal hidden unit activation function has the ability to produce linearly independent outputs. As a result, the minimum number of hidden units for a set of specified input is the number of patterns less the rank of the input patterns. In addition, the basis of many training algorithms is shown to inherently increase the number of linearly independent vectors in the internal representations, thereby increasing the likelihood of exact learning.
Related Concept Videos
Continuity for Functions of Multiple Variables
Multi-input and Multi-variable systems
In the absence of...
Linear time-invariant Systems
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be calculated...
Classification of Systems-I
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Associative Learning
Classical conditioning, also known...
Convolution Properties I
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output: