Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

45
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
45
Heuristics01:21

Heuristics

83
Heuristics are problem-solving strategies that use mental shortcuts to simplify decision-making. Unlike algorithms, which must be followed precisely to achieve a correct result, heuristics offer a general problem-solving framework. They save time and energy but can sometimes lead to less rational decisions.
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
83
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data01:16

Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data

117
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
117
Response Surface Methodology01:16

Response Surface Methodology

99
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
99
Parametric Survival Analysis: Weibull and Exponential Methods01:14

Parametric Survival Analysis: Weibull and Exponential Methods

390
Parametric survival analysis models survival data by assuming a specific probability distribution for the time until an event occurs. The Weibull and exponential distributions are two of the most commonly used methods in this context, due to their versatility and relatively straightforward application.
Weibull Distribution
The Weibull distribution is a flexible model used in parametric survival analysis. It can handle both increasing and decreasing hazard rates, depending on its shape parameter...
390
Statistical Methods to Analyze Parametric Data: ANOVA01:12

Statistical Methods to Analyze Parametric Data: ANOVA

302
Analysis of Variance, or ANOVA, is a powerful statistical technique used to analyze parametric data, primarily in research and experimental studies. It's designed to compare the means of two or more groups, assisting researchers in identifying any significant differences between these group means. There are two main types of ANOVA based on the complexity of the analysis: one-way and two-way.
One-way ANOVA is applied when a single independent variable or factor is scrutinized. It compares...
302

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Local Surrogate Models With Residual Fuzzy Rules for Model-Agnostic Explanations.

IEEE transactions on cybernetics·2026
Same author

SGFormer: Simplifying and Scaling Graph Transformers with Single-Layer Attention and Approximation-Free Linear Complexity.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

Hierarchical Mesh Representation Learning With Spectral Dictionary Embedding.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

Calibrating Biased Distribution in VFM-Derived Latent Space via Cross-Domain Geometric Consistency.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

Molecule generation for drug design: A graph learning perspective.

Fundamental research·2026
Same journal

Modeling the impact of budget limitation on the screening and treatment pathway of HPV-induced precancerous cervical lesions.

Mathematical biosciences and engineering : MBE·2026
Same journal

Modeling the effects of trait-mediated dispersal on coexistence of two species: Competition and non-consumptive predator-prey.

Mathematical biosciences and engineering : MBE·2026
Same journal

A close look at the viral reduction rate in target cell limited models.

Mathematical biosciences and engineering : MBE·2026
Same journal

A stochastic agent-based model for simulating tumor-immune dynamics and evaluating therapeutic strategies.

Mathematical biosciences and engineering : MBE·2026
Same journal

Addressing domain shift via imbalance-aware domain adaptation in embryo development assessment.

Mathematical biosciences and engineering : MBE·2026
Same journal

Effect of drug resistance on an HIV epidemic in heterogeneous populations.

Mathematical biosciences and engineering : MBE·2026
See all related articles

Related Experiment Video

Updated: Jun 15, 2025

Spatial Multiobjective Optimization of Agricultural Conservation Practices using a SWAT Model and an Evolutionary Algorithm
11:53

Spatial Multiobjective Optimization of Agricultural Conservation Practices using a SWAT Model and an Evolutionary Algorithm

Published on: December 9, 2012

12.9K

Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools.

Jia Mian Tan1, Haoran Liao1, Wei Liu1

  • 1Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China.

Mathematical Biosciences and Engineering : MBE
|August 23, 2024
PubMed
Summary
This summary is machine-generated.

This survey covers hyperparameter optimization (HPO) methods for machine learning. It details classical techniques, acceleration strategies, online HPO (dynamic algorithm configuration), and multi-objective optimization to improve model training efficiency and reduce costs.

Keywords:
bayesian optimizationdeep neural networkshyperparameter optimizationmachine learningsurvey

More Related Videos

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.4K
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.0K

Related Experiment Videos

Last Updated: Jun 15, 2025

Spatial Multiobjective Optimization of Agricultural Conservation Practices using a SWAT Model and an Evolutionary Algorithm
11:53

Spatial Multiobjective Optimization of Agricultural Conservation Practices using a SWAT Model and an Evolutionary Algorithm

Published on: December 9, 2012

12.9K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.4K
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.0K

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Computational Science

Background:

  • Hyperparameter optimization (HPO) is crucial for machine learning model performance.
  • Manual tuning is time-consuming, expertise-dependent, and hinders reproducibility.
  • Deep learning's rise has increased the need for efficient HPO.

Purpose of the Study:

  • To provide a comprehensive survey of hyperparameter optimization methods.
  • To categorize and explain different HPO approaches, including acceleration, online, and multi-objective settings.
  • To offer practical insights for researchers and practitioners.

Main Methods:

  • Survey of classical HPO techniques.
  • Categorization of acceleration strategies: multi-fidelity, bandit-based, early stopping.
  • Overview of dynamic algorithm configuration (DAC) methods: gradient-based, population-based, reinforcement learning.
  • Exploration of multi-objective HPO approaches: scalarization, metaheuristics, model-based algorithms.

Main Results:

  • Detailed review of established and advanced HPO methodologies.
  • Classification of techniques for optimizing HPO processes.
  • Discussion of frameworks and tools for practical HPO implementation.
  • Synthesis of methods for dynamic and multi-objective optimization scenarios.

Conclusions:

  • HPO is a vital and evolving research area in machine learning.
  • Various methods exist to address challenges in hyperparameter tuning.
  • The survey provides a valuable resource for understanding and applying HPO techniques.