The Compact Support Neural Network.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Neural networks often provide high-confidence predictions for out-of-distribution data, limiting their use in safety-critical applications.
• Existing models struggle with reliability when encountering data outside their training distribution.

Purpose of the Study:

• To introduce a novel neuron generalization with a shape parameter, bridging standard neurons and radial basis function (RBF) neurons.
• To develop a new training methodology for enhanced neural network reliability and performance.
• To theoretically and experimentally validate the proposed neuron's properties and effectiveness.

Main Methods:

• A novel neuron generalization is proposed, incorporating a shape parameter where standard and RBF neurons are extreme cases.
• A rectified linear unit (ReLU) activation function is used, resulting in a neuron with compact support.
• A progressive training method is introduced, fine-tuning pretrained networks while adjusting the shape parameter.

Main Results:

• Theoretical analysis provides bounds on the gradient and proves the universal approximation property of the proposed neural network.
• Experimental results on benchmark datasets show reduced test errors compared to state-of-the-art methods.
• The proposed approach demonstrates superior out-of-distribution sample detection on two out of three datasets.

Conclusions:

• The novel neuron generalization and training method enhance neural network reliability, particularly for safety-critical applications.
• The compact support property and universal approximation capability make the network robust and versatile.
• The findings suggest a significant advancement in developing more dependable AI systems.