Optimal Large Language Model Characteristics to Balance Accuracy and Energy Use for Sustainable Medical Applications.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Computational Sustainability

Background:

• Large language models (LLMs) in medicine have significant energy demands, contributing to healthcare's carbon footprint.
• The energy consumption and accuracy trade-offs of various LLMs for medical applications remain largely unquantified.

Purpose of the Study:

• To evaluate the relationship between accuracy and energy consumption across different types and sizes of LLMs for medical tasks.
• To identify optimal LLM configurations for balancing performance and environmental impact in healthcare.

Main Methods:

• Retrospective analysis of five sizes of two open-source LLMs (Llama 2 and Vicuna 1.5) using chest radiograph reports.
• Models were prompted to identify 13 CheXpert disease labels; energy use (kWh) and accuracy were measured.
• Efficiency ratios (accuracy per kWh) were calculated for each model.

Main Results:

• Vicuna 1.5 models (7B and 13B parameters) demonstrated superior efficiency ratios and higher labeling accuracy compared to Llama 2 models.
• The Vicuna 1.5 7B model achieved the highest efficiency ratio.
• Larger Llama 2 models (e.g., 70B) consumed substantially more energy with lower overall accuracy than smaller, fine-tuned models.

Conclusions:

• Smaller, fine-tuned LLMs (like Vicuna 1.5) offer a more sustainable approach for medical applications.
• LLM selection is critical for minimizing energy use in healthcare AI without sacrificing diagnostic accuracy.
• Specialized models present a viable path toward energy-efficient medical AI.