Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Reliability and Validity01:29

Reliability and Validity

12.8K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.8K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

708
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
708
Random and Systematic Errors01:20

Random and Systematic Errors

11.2K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
11.2K
Random and Systematic Errors01:20

Random and Systematic Errors

968
968
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

6.8K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.8K
Testing a Claim about Standard Deviation01:19

Testing a Claim about Standard Deviation

2.1K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.1K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Concordance of pathologic response assessment between local and central review in melanoma: a post hoc analysis of KEYMAKER-U02 substudy 02C.

ESMO open·2026
Same author

Sinonasal mucosal melanoma: REFCOR guidelines for diagnosis, treatment and follow-up.

European annals of otorhinolaryngology, head and neck diseases·2026
Same author

IO102-IO103 immune-modulatory cancer vaccine and pembrolizumab in melanoma.

Annals of oncology : official journal of the European Society for Medical Oncology·2026
Same author

Magnetic imaging under high pressure with a spin-based quantum sensor integrated in a van der Waals heterostructure.

Nature communications·2025
Same author

Neoadjuvant intralesional targeted immunocytokines (daromun) in stage III melanoma.

Annals of oncology : official journal of the European Society for Medical Oncology·2025
Same author

Impact of health simulation with OSCE on the acquisition of radiotherapy competencies by RTT students.

Radiography (London, England : 1995)·2025
Same journal

The role of digital resources in surgical education: An analysis of YouTube videos on dynamic stabilization.

Technology and health care : official journal of the European Society for Engineering and Medicine·2026
Same journal

Behavioral patterns in iGaming across territories: Psychiatric and AI-driven insights via the internet of behavior.

Technology and health care : official journal of the European Society for Engineering and Medicine·2026
Same journal

Leveraging personal health records for early heart failure risk prediction through AI-driven modeling.

Technology and health care : official journal of the European Society for Engineering and Medicine·2026
Same journal

From data to prevention: A systematic review of artificial intelligence applications in sports injury prediction.

Technology and health care : official journal of the European Society for Engineering and Medicine·2026
Same journal

Leadership styles and work outcome in healthcare sector: Insights from bibliometric analysis.

Technology and health care : official journal of the European Society for Engineering and Medicine·2026
Same journal

Network analysis revealing research focus of the German Congress of Orthopedics and Trauma Surgery 2021.

Technology and health care : official journal of the European Society for Engineering and Medicine·2026
See all related articles

Related Experiment Video

Updated: Apr 23, 2026

A Protocol of Manual Tests to Measure Sensation and Pain in Humans
07:28

A Protocol of Manual Tests to Measure Sensation and Pain in Humans

Published on: December 19, 2016

20.9K

A protocol for testing expert-system reliability.

P François1, C Robert2, G Weil2

  • 1Département de pédiatrie, CHU de Grenoble, 38700 La Tronche, France.

Technology and Health Care : Official Journal of the European Society for Engineering and Medicine
|October 3, 2014
PubMed
Summary
This summary is machine-generated.

This study introduces a protocol to validate computerised medical decision systems as expert systems. The method compares system agreement with experts and new physicians, using simple statistics for broad applicability in medical diagnosis.

Keywords:
AgreementConvex functionsEntropyExpert-systemKappa-coefficient

More Related Videos

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
10:39

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning

Published on: August 29, 2025

1.2K
Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

3.1K

Related Experiment Videos

Last Updated: Apr 23, 2026

A Protocol of Manual Tests to Measure Sensation and Pain in Humans
07:28

A Protocol of Manual Tests to Measure Sensation and Pain in Humans

Published on: December 19, 2016

20.9K
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
10:39

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning

Published on: August 29, 2025

1.2K
Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

3.1K

Area of Science:

  • Medical Informatics
  • Artificial Intelligence in Medicine
  • Clinical Decision Support

Background:

  • Computerised medical decision systems aim to achieve expert-level performance.
  • Validating these systems requires robust methodologies to ensure reliability and trustworthiness.

Purpose of the Study:

  • To establish a protocol for labeling computerised medical decision systems as "expert-systems."
  • To validate the MENINGE expert-system for pediatric meningitis diagnosis using the proposed protocol.

Main Methods:

  • A novel validation protocol comparing expert-system agreement with both medical experts and physicians new to practice.
  • Utilizes simple statistical computations, avoiding reliance on pre-defined correct therapeutic or diagnostic responses.
  • Recommends a standard agreement measure for inter-rater reliability assessment.

Main Results:

  • The protocol allows for broad application in validating diagnostic systems.
  • Demonstrates a method to compare system-clinician agreement against clinician-clinician agreement.
  • Highlights the importance of selecting an appropriate agreement measure.

Conclusions:

  • The proposed protocol offers a generalizable framework for validating expert medical systems.
  • The methodology facilitates objective assessment of diagnostic system performance against human expertise.
  • The study emphasizes the critical role of agreement metrics in system validation.