Generalised Medical Phrase Grounding

Area of Science:

• Radiology
• Medical Imaging
• Natural Language Processing

Background:

• Medical phrase grounding (MPG) links text descriptions to image regions for better report interpretation.
• Existing MPG systems often fail with complex reports containing multi-region findings or non-groundable phrases.

Purpose of the Study:

• To reformulate medical phrase grounding as a generalized task (GMPG) capable of handling complex radiological reports.
• To introduce MedGrounder, the first GMPG model, designed to map sentences to zero, one, or multiple image regions.

Main Methods:

• A two-stage training approach was employed: pre-training on sentence-anatomy box alignment and fine-tuning on human-annotated data.
• The MedGrounder model was evaluated on PadChest-GR and MS-CXR datasets.

Main Results:

• MedGrounder demonstrated strong zero-shot transfer capabilities and state-of-the-art performance on PadChest-GR.
• The model achieved significant improvements on multi-region phrases and required fewer human annotations.
• MedGrounder can be integrated with report generators to produce grounded reports.

Conclusions:

• Generalized medical phrase grounding (GMPG) offers a more robust approach to interpreting radiological reports.
• MedGrounder represents a significant advancement in accurately grounding complex radiological findings.
• The model's ability to handle diverse report structures enhances its clinical utility.