Deep Q-learning to globally optimize a k-D parameter search for medical imaging.

Area of Science:

• Computational Science
• Machine Learning
• Optimization

Background:

• Estimating global optima of multiple model parameters is crucial for characterizing physical environments and ensuring reproducible imaging.
• Nonconvex objective functions present challenges due to local minima, complicating parameter estimation.
• Global parameter searching can be framed as a k-dimensional move in parameter space, converting parameter updating into a state-action decision-making problem.

Purpose of the Study:

• To introduce a novel Deep Q-learning of Model Parameters (DQMP) method for global optimization of multiple parameters.
• To address the challenge of local minima in nonconvex objective functions for accurate parameter estimation.
• To improve the reliability and reproducibility of physical imaging through precise parameter extraction.

Main Methods:

• Proposed the Deep Q-learning of Model Parameters (DQMP) method for global optimization.
• Utilized a Deep Reward Network (DRN) with Long Short-Term Memory (LSTM) layers to learn global reward values from fitting and parameter errors.
• Modeled the k-dimensional parameter search as a state-action decision-making process, akin to a k-D board game.

Main Results:

• The DQMP method achieved relative errors of less than 4% in parameter estimation across various general functions.
• Compared to Q-learning (17% error) and Least Squares Method (21% error), DQMP demonstrated superior accuracy.
• Imaging experiments showed DQMP-estimated parameters produced images closest to ground truth simulations.

Conclusions:

• The DQMP method successfully achieves global optima, providing accurate model parameter estimates.
• DQMP shows significant promise for high-dimensional parameter estimation and complex nonconvex function optimization.
• The method is generalizable to various global optimization problems and physical parameter imaging.