Evaluating few-shot prompting for spectrogram-based lung sound classification using a multimodal language model.

Area of Science:

• Artificial Intelligence
• Biomedical Signal Processing
• Respiratory Medicine

Background:

• Traditional deep learning for lung sound analysis needs extensive labeled data.
• Multimodal large language models (LLMs) offer a potential prompt-based alternative.
• Evaluating general-purpose LLMs for respiratory sound classification is crucial.

Purpose of the Study:

• To assess GPT-4o's utility for lung sound classification using mel-spectrograms.
• To compare few-shot prompting against zero-shot prompting for this task.
• To establish a baseline for prompt-based multimodal inference in respiratory acoustics.

Main Methods:

• Converted 6898 annotated respiratory cycles from ICBHI 2017 database into mel-spectrograms.
• Employed GPT-4o with both zero-shot and few-shot prompting strategies for classification.
• Evaluated performance using accuracy, precision, recall, F1-score, and McNemar's test.

Main Results:

• Few-shot prompting yielded statistically significant improvements in accuracy (0.363 vs. 0.320) and other metrics.
• Model repeatability analysis showed high agreement (κ = 0.76-0.88), indicating excellent consistency.
• Performance gains were limited but statistically significant (p < 0.001).

Conclusions:

• GPT-4o shows potential for prompt-based lung sound classification, with few-shot prompting outperforming zero-shot.
• Current performance is insufficient for clinical deployment but provides a foundation for future research.
• Prompt-based multimodal inference offers a flexible approach for spectrogram analysis in respiratory medicine.