MedNLP-Hub: A knowledgebase platform for biomedical NLP tool discovery in clinical informatics

Biomedical natural language processing (NLP) tools are used to extract computable information from electronic health records, biomedical literature, and other health-related texts. However, selecting appropriate NLP tools for clinical informatics use remains difficult because available studies are fragmented across heterogeneous tasks, target variables, modeling strategies, and reporting practices. This study aimed to develop and evaluate MedNLP-Hub, a standardized knowledge-driven resource for multi-constraint discovery and comparative analysis of biomedical NLP tools.

Biomedical NLP studies published in PubMed up to March 2026 were systematically collected and curated. Each task-level record was annotated using a standardized metadata schema covering study characteristics, text sources, task types, target variables, terminology resources, modeling approaches, accessibility information, and evaluation metrics. Relevant metadata fields were normalized to support structured filtering and semantic retrieval. MedNLP-Hub was then evaluated using 18 task-oriented retrieval scenarios reflecting practical tool-selection needs in biomedical and clinical informatics. Performance was compared with PubMed and three general-purpose LLMs: ChatGPT 5.2, Gemini 2.5-Flash, and Claude Sonnet 4.6.

MedNLP-Hub contains 676 tools contributing 1,994 task-level instances, organized across 24 standardized metadata dimensions. The resource covers 5 NLP task categories, 23 languages, 4 modeling paradigms, and more than 3,200 normalized target-variable elements. The platform supports structured browsing, field-based filtering, semantic variable-level retrieval, tool comparison and visualization. In benchmark evaluations, MedNLP-Hub identified eligible tools for all predefined scenarios, whereas LLM-based systems and PubMed frequently failed to satisfy all structured constraints simultaneously.

MedNLP-Hub provides a standardized and clinically relevant infrastructure for biomedical NLP tool discovery. By combining structured metadata, variable normalization, and semantic retrieval, it supports transparent pre-implementation tool selection, methodological comparison, and reuse of NLP systems in biomedical research and clinical informatics workflows.