Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Assessing performance of prediction rules in machine learning.

Rory Martin1, Kai Yu

  • 1Millennium Pharmaceuticals, Cambridge MA 02139, USA. rory.martin.phd@gmail.com

Pharmacogenomics
|June 7, 2006
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Efficient homing of T cells via afferent lymphatics requires mechanical arrest and integrin-supported chemokine guidance.

Nature communications·2020
Same author

Carbon doped Fe<sub>3</sub>O<sub>4</sub> peroxidase-like nanozyme for mitigating the membrane fouling by NOM at neutral pH.

Water research·2020
Same author

Rapid Monitoring Approach for Microplastics Using Portable Pyrolysis-Mass Spectrometry.

Analytical chemistry·2020
Same author

Mutations in the HPV16 genome induced by APOBEC3 are associated with viral clearance.

Nature communications·2020
Same author

Power calculation for the general two-sample Mendelian randomization analysis.

Genetic epidemiology·2020
Same author

A Facile Grinding Method for the Synthesis of 3D Ag Metal-Organic Frameworks (MOFs) Containing Ag<sub>6</sub> Mo<sub>7</sub> O<sub>24</sub> for High-Performance Supercapacitors.

Chemistry (Weinheim an der Bergstrasse, Germany)·2020
Same journal

Closely similar tacrolimus morning troughs and graft function across the recipients' common <i>ABCB1</i> polymorphisms (<i>c.2677G>T/A</i>, <i>c.1236C>T</i>, <i>c.3435C>T</i>) early after renal transplantation.

Pharmacogenomics·2026
Same journal

Pharmacogenomics for stratified antidepressant treatment in major depressive disorder: evidence, limits and a roadmap for clinical use.

Pharmacogenomics·2026
Same journal

Guidance from an informed public on collecting and sharing PGx test results for major depressive disorder: "It's no different from your blood type".

Pharmacogenomics·2026
Same journal

Optimizing pharmacogenomics workflows: a qualitative evaluation of clinical service delivery models.

Pharmacogenomics·2026
Same journal

Combinations of single gene predictors with AI-driven polygenic precision therapies in cardiovascular diseases.

Pharmacogenomics·2026
Same journal

Practical guide to implementing pre-emptive pharmacogenetic screening in routine pediatric oncology care.

Pharmacogenomics·2026
See all related articles

Estimating prediction error is crucial for robust machine learning. The bootstrap method offers more accurate performance quantification than split sample or resubstitution techniques for reliable decision-making.

Area of Science:

  • Machine Learning
  • Statistical Modeling
  • Data Science

Background:

  • Assessing the robustness and replicability of machine learning prediction rules is vital for informed decision-making and future research.
  • Accurate estimation of a prediction rule's true error rate is necessary but challenging due to various resampling approaches.
  • This study addresses the need for practical guidance on selecting appropriate methods for true error estimation in machine learning.

Purpose of the Study:

  • To compare the performance of different methods for estimating true error in machine learning prediction rules.
  • To provide practical guidance on selecting the most effective error estimation techniques.
  • To illustrate the pros and cons of various resampling strategies for error estimation.

Main Methods:

Related Experiment Videos

  • Monte Carlo simulation studies were conducted to evaluate four error estimators: bootstrap, split sample, resubstitution, and direct estimation.
  • Stochastic gradient boosting was employed as the learning algorithm.
  • Data from two complex studies (anticancer drug design and cardiovascular) were utilized.

Main Results:

  • Significant differences were observed in the performance of the evaluated error estimators.
  • Split sample and resubstitution methods, though simple, were less accurate in quantifying prediction rule performance compared to the bootstrap.
  • These findings held true across both diverse study datasets.

Conclusions:

  • Adopting best practices for prediction error estimation can enhance the robustness and reliability of decisions derived from machine learning, particularly in genomics.
  • The bootstrap technique is recommended as a preferred method for reliable prediction error estimation.