IC neuron: An efficient unit to construct neural networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Neural networks are crucial for complex machine learning tasks, relying on neuron models for generalization.
• The McCulloch-Pitts (MP) neuron is a widely used model, but its structure is rarely optimized.
• Existing models may limit the representation capacity of neural networks.

Purpose of the Study:

• To introduce a novel neuron model, the Inter-layer Collision (IC) neuron, to enhance neural network capabilities.
• To improve the representation ability and non-linear transformation capacity of artificial neurons.
• To demonstrate the effectiveness of IC neurons in various network architectures and deep learning models.

Main Methods:

• Proposed the Inter-layer Collision (IC) neuron model, inspired by elastic collision principles in physics.
• Divided the input space into subspaces for distinct linear transformations within the IC neuron.
• Integrated IC neurons into fully-connected, convolutional, and recurrent neural network structures (IC networks).
• Evaluated IC networks and IC neuron integration with deep learning models on diverse tasks.

Main Results:

• IC neurons demonstrated enhanced non-linear representation ability by processing input space subspaces.
• IC networks showed superior performance compared to traditional neural networks across various tasks.
• Combining IC neurons with deep learning models yielded significant performance improvements.

Conclusions:

• The Inter-layer Collision (IC) neuron serves as an effective basic unit for constructing advanced neural network architectures.
• IC neurons significantly enhance the performance and representational power of machine learning models.
• This research opens new avenues for designing more capable and efficient neural network components.