On the complexity of computing and learning with multiplicative neural networks.

Area of Science:

• Computational neuroscience
• Machine learning theory

Background:

• Traditional neural networks sum inputs, limiting nonlinear interactions.
• Multiplicative neural networks utilize input multiplication for enhanced non-linearity.
• Existing research has explored higher-order and product unit networks.

Purpose of the Study:

• To introduce and analyze multiplicative neural networks.
• To investigate the computational complexity and learning capabilities of these networks.
• To derive bounds on Vapnik-Chervonenkis (VC) and pseudo-dimensions for multiplicative networks.

Main Methods:

• Derivation of upper and lower bounds for VC and pseudo-dimensions.
• Analysis of feedforward networks with product and sigmoidal units.
• Construction of product unit networks with super-linear VC dimension.
• Establishment of polynomial bounds for higher-order sigmoidal networks.

Main Results:

• Pseudo-dimension of general multiplicative networks is polynomially bounded, comparable to sigmoidal networks.
• Product unit networks demonstrate super-linear VC dimension.
• New polynomial bounds for higher-order sigmoidal networks are established without order restrictions.
• Asymptotically tight bounds are derived for sigma-pi units with connectivity constraints.

Conclusions:

• Multiplication enhances the computational power and learning capabilities of neural networks.
• The derived bounds offer new tools for assessing network complexity.
• This work broadens the understanding of neural network architectures beyond summation.