Proof of concept for voice based MRI scanner control using large language models in real time guided interventions

Area of Science:

• Medical Imaging
• Artificial Intelligence

Background:

• Clinical MRI-guided interventions often require dedicated assistants for scanner control due to equipment limitations.
• Real-time control of MRI scanners is crucial but challenging in interventional settings.

Purpose of the Study:

• To develop and evaluate a voice-based interaction system for hands-free MRI scanner control.
• To leverage large language models (LLMs) for natural language command execution in interventional MRI.

Main Methods:

• A multi-agent LLM system was developed for controlling MRI scanner functionalities.
• The system was tested in a real MRI scanning room for 18 predefined tasks, totaling 90 hands-free trials.

Main Results:

• The voice-based system achieved a 93.3% task completion rate in hands-free trials.
• Command response times ranged from 5 to 10.5 seconds on a consumer laptop without GPU support.

Conclusions:

• LLMs can feasibly enable voice-based interaction with MRI scanners during interventions.
• This system offers a potential solution to reduce reliance on human assistants and improve communication during procedures.