Beyond captions: linking figures with abstract sentences in biomedical articles

Area of Science:

• Biomedical Informatics
• Scientific Literature Analysis
• Information Retrieval

Background:

• Scientific figures contain crucial data but are underutilized in literature searches.
• Current systems often ignore figures or rely solely on caption text for retrieval.

Purpose of the Study:

• To develop and evaluate an automated method for associating figures within biomedical articles to specific sentences in their abstracts.
• To enhance the discoverability and accessibility of visual information within scientific publications.

Main Methods:

• Utilized supervised learning, including probabilistic language models, hidden Markov models, and conditional random fields.
• Integrated evidence from text within abstract sentences and figures, relative positioning, and cross-article association patterns.
• Employed a combination of text, positional, and pattern-based evidence for robust figure-sentence association.

Main Results:

• The most accurate method achieved an F1-score of 69% in cross-validation, outperforming existing state-of-the-art techniques.
• The system significantly improves access to figures linked to abstract sentences, reducing required mouse clicks by an average of 1.82.
• Performance was competitive with human expert accuracy.

Conclusions:

• The developed automated approach effectively links biomedical article figures with abstract sentences, enhancing information retrieval.
• The system offers a significant improvement over current methods and provides a user-beneficial tool for accessing visual research data.
• The system is accessible online for further research and application.