An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual...

Area of Science:

• Medical Informatics
• Clinical Trial Management
• Health Data Science

Background:

• Electronic Medical Records (EMRs) present challenges in clinical trial eligibility screening due to data complexity and varied terminologies.
• Manual screening is inefficient, requires expertise, and can lead to inconsistent participant selection, impacting patient safety and research outcomes, especially in critical situations like acute ischemic stroke.
• Existing computerized tools often require software engineering expertise for updates, limiting their practical use when eligibility criteria change.

Purpose of the Study:

• To develop and evaluate the intelligent trial eligibility screening tool (iTEST), which integrates natural language processing with a visual programming interface.
• To enable clinicians to independently create and modify eligibility screening rules.
• To assess the performance of iTEST's rule evaluation module against standard EMR interfaces.

Main Methods:

• An experiment was conducted with 12 clinicians at a tertiary teaching hospital using a 2-period crossover design.
• Clinicians evaluated the eligibility of stroke patients for two trials using both standard EMR and iTEST.
• iTEST utilized Google Blockly for rule authoring and MetaMap Lite for concept extraction from EMR data; outcomes included accuracy, task completion time, cognitive workload (NASA-TLX), and system usability (SUS).

Main Results:

• iTEST significantly improved accuracy (0.91 to 1.00, P<.001) and reduced completion time (3.18 to 2.44 min, P=.004) compared to standard EMR.
• Users reported lower cognitive workload (NASA-TLX: 39.7 vs 62.8, P=.02) and higher system usability (SUS: 71.3 vs 46.3, P=.01) with iTEST.
• Notable improvements in cognitive workload were seen in temporal demand, effort, and frustration.

Conclusions:

• iTEST demonstrated superior performance in clinical trial eligibility screening, enhancing accuracy, efficiency, and usability.
• Improved accuracy is crucial for patient safety, preventing inappropriate treatments or exclusion from beneficial trials.
• iTEST's adaptability to structured/unstructured data and ease of modification make it valuable for time-sensitive research and evolving protocols.