Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modeling.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Factorization Machines (FM) are effective for predictive analytics by learning feature interactions.
• Deep Neural Networks (DNNs) enhance FM expressiveness but lead to excessive parameters, hindering IoT/edge deployment.
• Current FM variants struggle with parameter efficiency for real-world applications.

Purpose of the Study:

• To propose novel, lightweight, and memory-efficient quaternion-valued models for sparse predictive analytics.
• To address the parameter inefficiency of deep FM variants for resource-constrained environments.
• To leverage hypercomplex quaternion representations for improved feature interaction modeling.

Main Methods:

• Introduced Quaternion Factorization Machine (QFM) and Quaternion Neural Factorization Machine (QNFM).
• Utilized quaternion algebra for feature interaction learning in a hypercomplex space.
• Compared QFM and QNFM against traditional FM and DNN-based FM variants.

Main Results:

• QFM improved performance by 4.36% over plain FM without extra parameters.
• QNFM outperformed baselines, achieving up to two orders of magnitude parameter reduction.
• The proposed quaternion models demonstrate significant efficiency gains and competitive performance.

Conclusions:

• Quaternion-based FM models offer a lightweight and memory-efficient alternative for sparse predictive analytics.
• These models are suitable for deployment on resource-constrained devices like IoT and edge systems.
• Hypercomplex representations provide a promising direction for enhancing FM-based predictive models.