Scalar Quantization as Sparse Least Square Optimization.

Area of Science:

• Machine Learning
• Optimization Techniques

Background:

• Scalar quantization is crucial for reducing neural network resource costs.
• Existing clustering-based methods face challenges like seed dependency and high complexity.

Purpose of the Study:

• To explore scalar quantization from a novel perspective using sparse least square optimization.
• To develop and implement new quantization algorithms addressing limitations of current techniques.

Main Methods:

• Proposed several quantization algorithms based on l1 least square optimization.
• Introduced similar schemes with l1 + l2 and l0 regularization.
• Developed iterative and clustering-based methods using sparse least square optimization.

Main Results:

• Algorithms were tested across three data scenarios, comparing computational performance.
• Evaluated information loss, time consumption, and sparse vector value distribution.
• Demonstrated superior performance in bit-width reduction scenarios.

Conclusions:

• Sparse least square optimization offers a new approach to quantization.
• The proposed algorithms show significant advantages, particularly for moderate bit-width reduction.