Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Classifier performance prediction for computer-aided diagnosis using a limited dataset.

Berkman Sahiner1, Heang-Ping Chan, Lubomir Hadjiiski

  • 1Department of Radiology University of Michigan, Ann Arbor Michigan 48109, USA. berki@umich.edu

Medical Physics
|May 22, 2008
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Physics-informed data augmentation to simulate low dose CT scans: Application to lung nodule detection.

Medical physics·2026
Same author

Bladder cancer segmentation using u-net-based deep-learning.

Scientific reports·2026
Same author

Synthetic data in radiological imaging: current state and future outlook.

BJR artificial intelligence·2026
Same author

Launching <i>BJR</i>|<i>Artificial intelligence</i>: an editorial.

BJR artificial intelligence·2026
Same author

Task-Based Sampling of Patient Data for Rigorous Machine Learning/AI Performance Assessment.

Journal of imaging informatics in medicine·2026
Same author

Quantifying the impact of a computer-aided diagnostic score on the clinical diagnosis of functional seizures.

Epilepsia·2026
Same journal

A novel optical respiratory gating system with a hybrid phase-amplitude algorithm for spot-scanning proton therapy.

Medical physics·2026
Same journal

Gamma Knife treatment planning using knowledge-based reinforcement learning.

Medical physics·2026
Same journal

Development and characterization of a novel, small animal external beam irradiator using a clinical high dose rate brachytherapy source.

Medical physics·2026
Same journal

Deep learning-based dose prediction for MR-guided prostate SIB: Supporting rapid feasibility assessment and adaptive editing margin selection.

Medical physics·2026
Same journal

Surface-guided analysis of breast shape changes during postoperative radiotherapy by using a functional map framework.

Medical physics·2026
Same journal

Monte Carlo assessment of a treatment planning system for intraoperative radiotherapy in the presence of tissue heterogeneities.

Medical physics·2026
See all related articles

For limited sample sizes in classifier design, the 0.632 and 0.632+ bootstrap methods offer the most accurate performance predictions. These resampling techniques minimize errors, especially in high-dimensional feature spaces, outperforming other methods like leave-one-out and ordinary bootstrap.

Area of Science:

  • Machine Learning
  • Statistical Modeling
  • Pattern Recognition

Background:

  • Classifier performance estimation is crucial in practical design, especially with limited data.
  • Resampling techniques are commonly used to predict classifier accuracy.
  • The true population is often unknown, necessitating reliable estimation methods.

Purpose of the Study:

  • To compare the effectiveness of various resampling techniques for classifier performance prediction.
  • To evaluate bias, variance, and root-mean-squared error (RMSE) of different methods.
  • To identify optimal resampling strategies for finite-sized datasets and high-dimensional feature spaces.

Main Methods:

  • A Monte Carlo simulation study was conducted.
  • Fisher's linear discriminant analysis was used as the classifier.

Related Experiment Videos

  • Resampling methods evaluated included Fukunaga-Hayes, leave-one-out, ordinary bootstrap, 0.632 bootstrap, and 0.632+ bootstrap.
  • Performance was measured by the area under the receiver operating characteristic curve (AUC) compared to the true AUC.

    • Main Results:

    • The 0.632 and 0.632+ bootstrap methods demonstrated the lowest RMSE, indicating superior accuracy.
    • These methods showed statistically smaller differences between estimated and true performances, particularly in high-dimensional spaces with small sample sizes.
    • The 0.632+ bootstrap method exhibited the lowest bias among the bootstrap techniques.

    Conclusions:

    • The 0.632 and 0.632+ bootstrap methods are recommended for classifier performance prediction with limited datasets.
    • These techniques provide more reliable estimates than ordinary bootstrap, leave-one-out, or Fukunaga-Hayes methods under tested conditions.
    • The findings highlight the importance of selecting appropriate resampling strategies for accurate classifier evaluation.