Physarum Powered Differentiable Linear Programming Layers and Applications.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Optimization

Background:

• Integrating optimization routines (e.g., eigenvalue problems, cone programming) as layers in deep neural networks (DNNs) is challenging.
• Recent advancements enable differentiable eigendecomposition and sorting within DNNs.

Purpose of the Study:

• To develop an efficient and differentiable solver for general linear programming (LP) problems.
• To enable seamless integration of LP solvers as layers in DNNs.

Main Methods:

• The solver is inspired by the optimization dynamics observed in slime mold (physarum).
• It functions as a plug-and-play layer within deep neural networks.
• The method was applied to video segmentation and few-shot meta-learning tasks.

Main Results:

• The proposed LP solver demonstrates comparable performance to projected gradient descent in video segmentation.
• It outperforms the differentiable CVXPY-SCS solver in few-shot meta-learning.
• The solver exhibits rapid convergence without requiring a feasible initial point.

Conclusions:

• The developed solver offers an efficient and easy-to-implement solution for incorporating LP into DNNs.
• It provides a fast, approximate solution for LP problems within larger neural network architectures.
• The approach is versatile and applicable to various learning procedures requiring optimization layers.