PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Hypercomplex neural networks leverage Clifford algebras to reduce parameters while maintaining performance.
• Recent advancements include efficient parameterized Kronecker products in hypercomplex linear layers.

Purpose of the Study:

• Introduce Parameterized Hypercomplex Neural Networks (PHNNs) as lightweight and efficient large-scale models.
• Define parameterization for hypercomplex convolutional layers.
• Enable flexible operation across various dimensions (1-D to N-D) without predefined domain structures.

Main Methods:

• Develop parameterization for hypercomplex convolutional layers.
• Introduce the family of Parameterized Hypercomplex Neural Networks (PHNNs).
• Design models that learn convolution rules and filter organization directly from data.

Main Results:

• PHNNs operate with significantly fewer parameters (1/n) compared to real-valued counterparts.
• Demonstrated superior performance over real and quaternion-valued networks on image and audio datasets.
• Showcased flexibility in processing multidimensional data in its natural domain.

Conclusions:

• PHNNs offer a versatile and parameter-efficient alternative for deep learning applications.
• The proposed method effectively handles multidimensional data without artificial dimension expansion.
• PHNNs represent a significant advancement in developing lightweight and high-performing neural network architectures.