Overview of the COVID-19 text mining tool interactive demonstration track in BioCreative VII

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Scientific Communication

Background:

• The COVID-19 pandemic necessitated rapid data sharing for medical treatments and public health.
• Preprint publications emerged as key sources for timely scientific results, lacking peer review validation.
• Natural Language Processing (NLP) systems are increasingly used to extract and organize COVID-19 data.

Purpose of the Study:

• To assess the landscape and user interest in COVID-19 text mining tools.
• To create a communication channel between NLP system developers and end-users.
• To inform system designers on tool performance, usability, and potential new features.

Main Methods:

• The BioCreative COVID-19 text mining tool interactive demonstration track was established.
• Seven teams submitted NLP systems for COVID-19 related tasks.
• Over 30 volunteer users from diverse scientific backgrounds tested the systems and provided feedback via surveys.

Main Results:

• The track successfully gathered feedback from both NLP system developers and volunteer users.
• Users evaluated system performance and usability, with options for anonymous participation.
• Participating teams and users reported positive reception of the track.

Conclusions:

• The BioCreative COVID-19 track effectively evaluated NLP tools for COVID-19 data extraction.
• The initiative fostered valuable two-way communication between tool developers and the research community.
• Positive feedback indicates the utility of such tracks for advancing biomedical text mining.