Enhancing biomedical search interfaces with images

Area of Science:

• Biomedical Informatics
• Information Retrieval
• Computer Vision

Background:

• Biomedical literature contains crucial information within figures, yet current search engines primarily rely on text.
• Effective retrieval of biomedical documents is essential for research and clinical practice.

Purpose of the Study:

• To develop and evaluate a search system that integrates image modalities and figure-based data for enhanced biomedical document retrieval.
• To improve the precision and recall of searching biomedical documents by leveraging visual content.

Main Methods:

• Developed a custom search system integrating a taxonomy of image modalities and image-based data.
• Implemented a front-end interface to augment traditional search results with figure thumbnails, captions, and image-modality information.
• Demonstrated the system on the CORD-19 document collection.

Main Results:

• Quantitative evaluation showed higher precision and recall for biomedical document retrieval using the image-enhanced system.
• Qualitative evaluation with domain experts confirmed the system's benefits for biomedical search.
• The system successfully integrates image modalities and figure data into the search process.

Conclusions:

• Leveraging image modalities and figure data significantly improves biomedical document search.
• The developed system offers a powerful new tool for researchers and clinicians seeking relevant biomedical information.
• Future work includes continuous dataset expansion and model refinement.