Fair human-centric image dataset for ethical AI benchmarking

Area of Science:

• Computer Vision
• Artificial Intelligence (AI)
• Machine Learning Ethics

Background:

• AI and computer vision rely on vast datasets, often collected without ethical considerations, leading to biased and non-diverse data.
• Existing datasets perpetuate biases and lack consent, compromising AI model fairness, accuracy, and stakeholder rights.
• A significant gap exists in publicly available, ethically sourced datasets for evaluating bias in computer vision tasks.

Purpose of the Study:

• Introduce the Fair Human-Centric Image Benchmark (FHIBE), a novel, ethically curated human image dataset.
• Provide a resource for evaluating and mitigating bias in various human-centric computer vision applications.
• Establish best practices for responsible data collection and curation in AI.

Main Methods:

• Developed FHIBE with a focus on consent, privacy, compensation, safety, diversity, and utility.
• Implemented comprehensive annotations including demographic, physical, environmental, and pixel-level attributes.
• Designed FHIBE for use in fairness evaluations across tasks like pose estimation, segmentation, and face recognition.

Main Results:

• FHIBE offers a publicly available, ethically sourced benchmark for AI fairness.
• The dataset's detailed annotations enable identification of diverse biases.
• FHIBE facilitates nuanced bias diagnosis for improved AI model development.

Conclusions:

• FHIBE represents a significant advancement in creating trustworthy AI.
• The benchmark raises the standard for fairness evaluations in computer vision.
• FHIBE provides a roadmap for responsible data curation in the AI field.