Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Parallel Processing01:20

Parallel Processing

157
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
157
Distributed Loads: Problem Solving01:21

Distributed Loads: Problem Solving

650
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
650
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models01:06

Model Approaches for Pharmacokinetic Data: Distributed Parameter Models

74
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
74
Improving Translational Accuracy02:07

Improving Translational Accuracy

11.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.3K
Distributed Loads01:19

Distributed Loads

541
Distributed loads are a common type of load that engineers and scientists encounter in various practical situations. Distributed loads often refer to a type of load spread over a surface or a structure and can be modeled as continuous force per unit area.
For example, consider a bookshelf filled with books stacked vertically adjacent to each other. The weight of the books is evenly distributed over the length of the shelf. As a result, the pressure at different locations on the surface of the...
541
Multicompartment Models: Overview01:14

Multicompartment Models: Overview

151
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
151

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Risk factors for severe COVID-19 outcomes in the Asia-Pacific region: a literature review.

Frontiers in public health·2025
Same author

Improving Voluntary Compliance through Problem-Oriented Governance: A Case Study of South Korea's Efforts to Increase the COVID-19 Vaccination Rate.

Journal of Asian and African studies·2024
Same author

Development and operation of a digital platform for sharing pathology image data.

BMC medical informatics and decision making·2021
Same author

Modified dose of melphalan-prednisone in multiple myeloma patients receiving bortezomib plus melphalan-prednisone treatment.

The Korean journal of internal medicine·2018
Same author

A Wearable Wrist Band-Type System for Multimodal Biometrics Integrated with Multispectral Skin Photomatrix and Electrocardiogram Sensors.

Sensors (Basel, Switzerland)·2018
Same author

A comparative study of proteomic differences between pencil and storage roots of sweetpotato (Ipomoea batatas (L.) Lam.).

Plant physiology and biochemistry : PPB·2015
Same journal

Analysis of strength degradation of coal and rock masses and stability of mined areas under long term immersion environment.

PloS one·2026
Same journal

Biogenic Silver-Selenium nanocomposite with anticancer activity and potent efficacy against vancomycin-resistant Staphylococcus aureus.

PloS one·2026
Same journal

Preparation and physicochemical characterization of a biodegradable chitosan/carboxymethyl cellulose hydrogel synthesized in NaOH/urea medium.

PloS one·2026
Same journal

Action-guilt, survivor-guilt, and depression in combat-related PTSD.

PloS one·2026
Same journal

Explainable machine learning for predicting activities of daily living at discharge in stroke patients: A retrospective study using SHAP interpretability.

PloS one·2026
Same journal

Deep learning based two-way feature depiction model for brain tumor detection.

PloS one·2026
See all related articles

Related Experiment Video

Updated: Jul 11, 2025

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.2K

Towards accelerating model parallelism in distributed deep learning systems.

Hyeonseong Choi1, Byung Hyun Lee2, Se Young Chun2,3

  • 1Department of Computer Engineering, Korea Aerospace University, Goyang, South Korea.

Plos One
|November 2, 2023
PubMed
Summary
This summary is machine-generated.

Training large deep neural networks on multiple GPUs is optimized by finding the best micro-batch size for efficient pipelining and using appropriate normalization techniques for model and data parallelism.

More Related Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

596
Dorsal Column Steerability with Dual Parallel Leads using Dedicated Power Sources: A Computational Model
11:19

Dorsal Column Steerability with Dual Parallel Leads using Dedicated Power Sources: A Computational Model

Published on: February 10, 2011

11.9K

Related Experiment Videos

Last Updated: Jul 11, 2025

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.2K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

596
Dorsal Column Steerability with Dual Parallel Leads using Dedicated Power Sources: A Computational Model
11:19

Dorsal Column Steerability with Dual Parallel Leads using Dedicated Power Sources: A Computational Model

Published on: February 10, 2011

11.9K

Area of Science:

  • * Deep Learning
  • * Distributed Systems
  • * High-Performance Computing

Background:

  • * Training large deep neural networks (DNNs) often requires multiple GPUs due to model and data size.
  • * Model parallelism, splitting models across GPUs, faces challenges in scalability and efficiency due to communication overhead.
  • * Inefficient pipelining and normalization methods can hinder GPU utilization and model accuracy in distributed training.

Purpose of the Study:

  • * To investigate efficient pipelining and normalization techniques for distributed DNN training across multiple GPUs.
  • * To address challenges in model parallelism, aiming to maximize GPU utilization and maintain model accuracy.
  • * To enable training of large models with large mini-batches without compromising performance.

Main Methods:

  • * Developed a novel algorithm to search for optimal micro-batch sizes tailored to GPU count and memory for model parallelism.
  • * Investigated the impact of different normalization methods (Batch Normalization, Group Normalization) on distributed training performance.
  • * Conducted experiments comparing proposed methods against conventional model parallelism for efficiency and accuracy.

Main Results:

  • * The proposed micro-batch size search algorithm increased image throughput by up to 12% and trainable mini-batch size by 25%.
  • * Sharing batch information improved Batch Normalization performance in data parallelism.
  • * Group Normalization minimized accuracy degradation in pipelined model parallelism and ensured consistent accuracy across various mini-batch sizes.

Conclusions:

  • * Optimal micro-batch sizing is crucial for efficient pipelining and maximizing GPU utilization in model parallelism.
  • * Group Normalization is effective in mitigating accuracy loss during pipelined model parallelism.
  • * The study provides practical solutions for enhancing the scalability and accuracy of distributed deep learning training.