Quadratic unconstrained binary optimization formulation for rectified-linear-unit-type functions.

Area of Science:

• Optimization Theory
• Machine Learning
• Quantum Computing

Background:

• Rectified-linear-unit (ReLU) functions are fundamental in modern machine learning.
• Existing methods like q-loss have limitations in formulating ReLU functions for optimization.
• Quadratic Unconstrained Binary Optimization (QUBO) is a powerful framework for combinatorial optimization problems.

Purpose of the Study:

• To propose a novel QUBO formulation for ReLU-type functions.
• To extend the applicability of QUBO to neural network activation functions.
• To enable the use of advanced optimization techniques, including quantum annealing, for ReLU-based models.

Main Methods:

• Developed a QUBO formulation for ReLU-type functions.
• Utilized both Legendre duality and Wolfe duality in the derivation.
• Demonstrated the formulation's compatibility with Ising-type annealing methods.

Main Results:

• Successfully derived a QUBO formulation for ReLU-type functions.
• The proposed formulation is distinct from previous q-loss approaches.
• The QUBO formulation is directly applicable to quantum annealing hardware.

Conclusions:

• The novel QUBO formulation provides a new way to represent ReLU functions.
• This work bridges the gap between deep learning activation functions and quantum optimization.
• The findings open avenues for enhanced computational efficiency in machine learning tasks using quantum annealing.