A dot product neuron for hardware implementation of competitive networks

Area of Science:

• Computational Neuroscience
• Machine Learning

Background:

• Competitive models often rely on dot product neurons requiring computationally expensive vector normalization.
• Euclidean distance nodes offer an alternative but have biological implausibility and higher computational costs than dot products.

Purpose of the Study:

• To propose a novel dot product neuron model that avoids vector normalization.
• To introduce an incremental learning rule for the proposed neuron.
• To enhance the suitability of competitive networks for hardware implementation.

Main Methods:

• The proposed neuron model is formally equivalent to a Euclidean neuron.
• It requires subtracting an iteratively computed bias from the dot product.
• A simple incremental learning rule is developed for the neuron.

Main Results:

• The novel dot product neuron eliminates the need for vector normalization.
• The computational cost is reduced compared to traditional normalized dot product neurons and Euclidean distance nodes.
• The model is compatible with hardware implementation.

Conclusions:

• The proposed dot product neuron offers a computationally efficient and biologically plausible alternative for competitive networks.
• The associated incremental learning rule simplifies network training.
• This model is well-suited for efficient hardware implementation of competitive neural networks.