Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

3DAD: Super-Resolution Image Synthesis from Anisotropic CT Images Using a Three-Dimensional Adversarial Diffusion Model.

Bioengineering (Basel, Switzerland)·2026

Same author

Scalable and Robust Artificial Intelligence for Spine Alignment Assessment: Multicenter Study Enabled by Real-Time Data Transformation.

Journal of medical Internet research·2026

Same author

Robust Normal Estimation for 3D LiDAR Point Clouds in Urban Environments.

Sensors (Basel, Switzerland)·2019

Same journal

Latent Space Projections and Atlases, a Cautionary Tale in Deep Neuroimaging using Autoencoders.

International journal of neural systems·2026

Same journal

Transformer-Based Anomaly Detection for Neurodegenerative Screening in MRI Images.

International journal of neural systems·2026

Same journal

Discrete Wavelet Convolution for Learnable Time-Frequency Representation with Application to Seizure Prediction.

International journal of neural systems·2026

Same journal

Automatic Seizure Detection using Hierarchical Spectral-Temporal Feature Learning with an Imbalance-Aware Transformer.

International journal of neural systems·2026

Same journal

Pyramid Vision Transformer-Enhanced Conformer Network for Epileptic Seizure Recognition Using MultiChannel EEG Signals.

International journal of neural systems·2026

Same journal

A Time-Frequency Decoupled Contrastive Learning Framework for Electroencephalography-Based Parkinson's Disease Diagnosis.

International journal of neural systems·2026

See all related articles

Search research articles

Related Experiment Video

Updated: Jul 4, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Automated Quality Evaluation of Large-Scale Benchmark Datasets for Vision-Language Tasks.

Ruibin Zhao^1,2, Zhiwei Xie¹, Yipeng Zhuang¹

¹Department of Mathematics and Information Technology, The Education University of Hong Kong, Hong Kong SAR, P. R. China.

International Journal of Neural Systems

|February 6, 2024

Summary

This summary is machine-generated.

This study introduces an automated method to evaluate vision-language benchmark datasets. Findings reveal significant quality variations in ground-truth descriptions, with some being unreliable for AI model training.

Keywords:

Benchmark datasets automated scoring cross-modal deep learning quality evaluation vision-language tasks

More Related Videos

Author Spotlight: Insights into Visual Cortex Research Through Wide-View fMRI Mapping

Author Spotlight: Insights into Visual Cortex Research Through Wide-View fMRI Mapping

Published on: December 8, 2023

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Published on: March 17, 2023

Related Experiment Videos

Last Updated: Jul 4, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Author Spotlight: Insights into Visual Cortex Research Through Wide-View fMRI Mapping

Author Spotlight: Insights into Visual Cortex Research Through Wide-View fMRI Mapping

Published on: December 8, 2023

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Published on: March 17, 2023

Area of Science:

Computer Science
Artificial Intelligence
Machine Learning

Background:

Large-scale benchmark datasets are vital for advancing AI model development and performance evaluation.
Existing vision-language datasets like Flickr30k, COCO, and NoCaps pair images with textual descriptions.

Purpose of the Study:

To propose an automatic method for assessing the quality of large-scale benchmark datasets for vision-language tasks.
To identify potential issues with the reliability of ground-truth descriptions in these datasets.

Main Methods:

Development of a novel cross-modal matching model to automatically score textual descriptions against visual images.
Application of the developed model to evaluate existing vision-language datasets by scoring each image-description pair.

Main Results:

The automated scoring method shows good agreement with manual evaluations.
Significant disparities in the quality of ground-truth descriptions across benchmark datasets were identified.
A notable portion of descriptions were found to be unsuitable as reliable ground-truth references.

Conclusions:

The proposed automated method effectively assesses vision-language dataset quality.
There is a critical need for careful scrutiny and utilization of publicly available benchmark datasets.
Improving dataset quality is essential for the reliable progress of AI systems in vision-language research.