Related Experiment Videos
Development Parameters of the Decision Aid Rule-Based Evaluation and Support Tool (REST) for Optimizing the Resources
Guillaume Bazin1, Akram Redjdal2, Xavier Tannier1
1Sorbonne University, Inserm, Sorbonne Paris Nord University, Limics, Paris, France.
Studies in Health Technology and Informatics
|May 23, 2026
Summary
We developed REST, a decision tool aiding developers in selecting information extraction (IE) methods. It defaults to rules-based IE for efficiency and interpretability, suggesting machine learning when needed.
Area of Science:
- Natural Language Processing (NLP)
- Bioinformatics
- Machine Learning
Background:
- Information Extraction (IE) is crucial for processing unstructured text data.
- Selecting the optimal IE method can be challenging for developers without NLP expertise.
- Existing methods often lack clear guidance on choosing between rule-based and machine learning approaches.
Purpose of the Study:
- To develop and validate a decision aid tool (REST) for selecting appropriate Information Extraction (IE) methods.
- To guide Natural Language Processing (NLP) agnostic developers in choosing between rule-based and supervised machine learning (ML) approaches.
- To establish criteria for defaulting to rule-based IE based on performance, linguistic homogeneity, and data occurrence.
Main Methods:
- Developed the REST (Rule-based Extraction System Tool) decision aid.
- Defined feasibility rules for rule-based IE based on performance metrics (sensitivity, positive predictive value), linguistic homogeneity (>10%), and entity occurrence (>25%).
- Tested REST on 12 entities across 35 medical records with 2 oncologists, assessing external validity.
Main Results:
- REST suggests rule-based IE when sensitivity and positive predictive value exceed 75%, linguistic homogeneity is >10%, and occurrence is >25%.
- Otherwise, supervised ML is recommended.
- External validation demonstrated a high agreement rate of 91.7%.
Conclusions:
- The REST tool effectively aids NLP-agnostic developers in assessing the feasibility of rule-based IE.
- REST promotes efficient and interpretable IE by prioritizing rule-based methods as the default.
- Supervised ML serves as a viable backup, with Large Language Models (LLMs) under investigation for future enhancements.
Related Concept Videos
01:09Decision Making: P-value Method
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
01:16Response Surface Methodology
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
01:14Decision Making: Traditional Method
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...