Visual-language artificial intelligence system for knee radiograph diagnosis and interpretation: a collaborative...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology
• Machine Learning for Diagnostics

Background:

• Large language models (LLMs) show promise in text-based clinical tasks but cannot interpret medical images like knee radiographs.
• Current AI tools lack inherent capabilities for direct medical image interpretation, necessitating new approaches.

Purpose of the Study:

• To develop an interactive diagnostic approach, Radiology Generative Pretrained Transformer (RadGPT), for knee radiological image interpretation.
• To assist and synergize with human users in diagnosing knee conditions from radiological images.

Main Methods:

• Utilized 22,512 knee roentgenograms and reports from Massachusetts General Hospital (80% training, 10% testing/validation).
• Selected 15 high-frequency, clinically relevant diagnostic features (e.g., osteoarthritis, effusion) for image labeling.
• Employed metrics like BiLingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) to assess diagnostic performance and text generation quality.

Main Results:

• RadGPT achieved Area Under the Curve (AUC) scores from 0.76 (osteonecrosis) to 0.91 (arthroplasty) across 15 diagnostic categories.
• Demonstrated significantly higher performance than baseline LLM methods in BLEU (0.18), ROUGE-L (0.30), METEOR (0.10), and SCPS (0.15) scores.
• Exhibited good linguistic overlap and clinical consistency with reference reports.

Conclusions:

• RadGPT shows advanced capabilities in knee radiograph feature recognition, highlighting LLMs' potential in medical image interpretation.
• The study provides a training protocol for developing AI-assisted tools for knee radiological image diagnosis and interpretation.
• This human-AI interactive approach signifies a step forward in augmenting radiological diagnostic workflows.