Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Video

Updated: May 31, 2026

PIPEMAT-RS: Development and Validation of a Standardized MATLAB Pipeline for Resting-State EEG Preprocessing

PIPEMAT-RS: Development and Validation of a Standardized MATLAB Pipeline for Resting-State EEG Preprocessing

Published on: June 6, 2025

PipeBench: a benchmarking framework for end-to-end machine learning pipelines.

Sanjay Agal¹, Ruchika Katariya²

¹Department of Artificial Intelligence and Data Science, Faculty of Engineering and Technology, Parul University, Vadodara, Gujarat, India. sanjay.agal32685@paruluniversity.ac.in.

Scientific Reports

|May 28, 2026

Summary

This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

A machine learning framework for early warning prediction of student success using privacy preserving synthetic educational data.

Scientific reports·2026

Same author

A machine learning framework for structural and predictive analysis of intelligent data networks.

Scientific reports·2026

Same author

A machine learning framework with a public synthetic data set for early student profiling and outcome prediction.

Scientific reports·2026

Same author

A privacy preserving synthetic learner dataset for learning analytics in technology enhanced higher education.

Scientific reports·2026

Same author

A machine learning approach to risk based asset allocation in portfolio optimization.

Scientific reports·2025

Same journal

A tri-axis optomechanical accelerometer with plasmonic MIM waveguide and structural direction-dependent optical signatures.

Scientific reports·2026

Same journal

Holographic leaky-wave antennas with independently controlled multiple counter-rotating vortex beams.

Scientific reports·2026

Same journal

Differential associations of longitudinal hearing and vision trajectories with dementia and mild cognitive impairment in older adults.

Scientific reports·2026

Same journal

Abdominal obesity and leisure-time sedentary behavior in relation to gastroesophageal reflux disease risk: a prospective cohort study from the UK Biobank.

Scientific reports·2026

Same journal

Effect of nitrogen-rich COF incorporation on the structure and separation performance of polyamide nanofiltration membranes.

Scientific reports·2026

Same journal

Withanolide A inhibits hIAPP aggregation: An In silico, biophysical, and drosophila-based In vivo validation.

Scientific reports·2026

See all related articles

PipeBench evaluates end-to-end machine learning (ML) pipelines, revealing model training as the primary latency driver. Optimized AutoML strategies significantly reduce computational costs, enhancing ML system efficiency.

Area of Science:

Computer Science
Machine Learning Systems
Data Engineering

Background:

Modern data management systems increasingly integrate machine learning (ML) for intelligent decision-making.
Existing performance benchmarks often assess isolated ML components, neglecting the complexities of full end-to-end pipelines.
A comprehensive understanding of integrated ML pipeline behavior is crucial for optimizing large-scale information infrastructures.

Purpose of the Study:

To introduce PipeBench, a reproducible framework for evaluating integrated machine learning pipelines.
To analyze the performance of pipelines encompassing distributed data processing, automated machine learning (AutoML), model management, and production serving.
To provide practical design guidelines for scalable ML-enabled information systems.

Main Methods:

Keywords:

AutoML Benchmarking Distributed systems Machine learning pipelines Performance evaluation Reproducibility

Related Experiment Videos

Last Updated: May 31, 2026

PIPEMAT-RS: Development and Validation of a Standardized MATLAB Pipeline for Resting-State EEG Preprocessing

PIPEMAT-RS: Development and Validation of a Standardized MATLAB Pipeline for Resting-State EEG Preprocessing

Published on: June 6, 2025

Developed PipeBench using open-source technologies: Apache Spark, Kubeflow, MLflow, and TensorFlow Serving.
Conducted experiments on diverse datasets (industrial IoT, e-commerce, financial analytics) varying data scale, cluster size, and AutoML strategies.
Evaluated pipeline latency, computational cost, and accuracy of different AutoML approaches, including Bayesian optimization and BOHB.

Main Results:

Model training constitutes the largest portion of pipeline latency (58-70%), followed by preprocessing (19-28%).
Bayesian optimization achieved 97% of evolutionary AutoML accuracy at nearly half the computational cost.
The multi-fidelity approach BOHB further reduced cost by 43% compared to standard Bayesian optimization for equivalent accuracy.
Identified system bottlenecks: GPU underutilization, memory pressure from shuffling, and storage contention during checkpointing.

Conclusions:

PipeBench offers a standardized framework for reproducible benchmarking of integrated ML pipelines.
Efficient AutoML strategies and addressing system bottlenecks are key to optimizing ML-enabled information systems.
The study provides valuable insights and practical guidelines for building scalable and performant ML infrastructures.