Criteria2Query: a natural language interface to clinical databases for cohort definition

Area of Science:

• Clinical Informatics
• Natural Language Processing
• Biomedical Research

Background:

• Cohort definition is a major bottleneck in clinical research, often relying on subjective expert decisions.
• Data-driven cohort definition requires significant expertise in terminologies and clinical data models.
• Existing methods lack efficient, user-friendly tools for translating complex criteria into executable queries.

Purpose of the Study:

• To introduce Criteria2Query, a novel natural language interface for human-computer collaboration in clinical cohort definition.
• To facilitate the transformation of unstructured eligibility criteria into structured, executable clinical data queries.
• To enable researchers to define cohorts autonomously with minimal effort.

Main Methods:

• Criteria2Query employs a hybrid information extraction pipeline combining machine learning and rule-based approaches.
• Eligibility criteria text is systematically parsed into a structured representation and then translated into SQL queries compatible with the OMOP Common Data Model.
• The system was evaluated on 125 criteria from ClinicalTrials.gov and 52 user-entered criteria, assessing F1 scores, accuracy, computation time, and user satisfaction.

Main Results:

• Achieved F1 scores of 0.795 for entity recognition and 0.805 for relation extraction.
• Demonstrated high accuracy in negation (0.984) and logic detection (0.864), with notable performance in entity (0.514) and attribute normalization (0.793).
• Fully automated query formulation averaged 1.22 seconds per criterion, and over 80% of users indicated willingness to use the tool in the future.

Conclusions:

• Criteria2Query offers a novel, open-source natural language interface for clinical databases.
• The tool supports both fully automated and interactive modes for autonomous, data-driven cohort definition.
• Criteria2Query demonstrates promising user-friendliness and usability, significantly reducing human effort in cohort definition.