A Heuristically Accelerated Reinforcement Learning-Based Neurosurgical Path Planner.

Area of Science:

• Neurosurgery
• Medical Robotics
• Artificial Intelligence

Background:

• Steerable needles offer enhanced maneuverability in neurosurgery, enabling bypass of critical brain regions and minimizing damage through optimized path planning.
• Reinforcement learning (RL) based path planning shows promise but suffers from computational expense, low training efficiency, and safety concerns due to its trial-and-error nature.

Purpose of the Study:

• To develop a safe and efficient algorithm for preoperative planning of steerable needle insertion paths in neurosurgical environments.
• To improve the training efficiency and reduce the computational cost associated with RL-based path planning in neurosurgery.

Main Methods:

• A heuristically accelerated deep Q network (DQN) algorithm was proposed for needle path planning.
• A fuzzy inference system was integrated to balance heuristic policy and RL algorithm for improved performance and safety.
• Simulations were conducted to compare the proposed method against traditional greedy heuristic searching and standard DQN algorithms.

Main Results:

• The proposed algorithm demonstrated significant improvements, saving over 50 training episodes compared to existing methods.
• Normalized path lengths were reduced to 0.35, outperforming DQN (0.61) and greedy heuristic search (0.39).
• Maximum planning curvature was decreased from 0.139 mm⁻¹ to 0.046 mm⁻¹, indicating a safer and smoother needle trajectory.

Conclusions:

• The heuristically accelerated DQN with fuzzy inference offers a safe, efficient, and computationally effective solution for steerable needle path planning in neurosurgery.
• This approach enhances preoperative planning by optimizing needle trajectories, reducing potential damage, and improving training efficiency for robotic-assisted neurosurgical interventions.