Responsible AI measures dataset for ethics evaluation of AI systems.

Area of Science:

• Artificial Intelligence
• Computer Science
• Ethics

Background:

• Effective governance of Artificial Intelligence (AI) systems necessitates robust assessment and monitoring.
• Global efforts are establishing AI ethical principles such as fairness, transparency, and privacy for governance.
• Existing measures for AI normative qualities are often fragmented, principle-specific, or limited in scope.

Purpose of the Study:

• To address the fragmentation of AI ethical measures by creating a consolidated dataset.
• To enable practitioners to explore and critically analyze current approaches to measuring AI normative qualities.

Main Methods:

• Extracted 12,067 data points on 791 evaluation measures for 11 ethical principles.
• Compiled data from a corpus of 257 computing literature publications (2011-2023).
• Included detailed descriptions of measures, AI system characteristics, and publication metadata.

Main Results:

• Developed the Responsible AI Measures Dataset, a comprehensive collection of AI ethical assessment methods.
• The dataset covers 11 key ethical principles and includes extensive metadata.
• An interactive visualization tool was created to enhance data usability and interpretation.

Conclusions:

• The Responsible AI Measures Dataset provides a centralized resource for understanding AI ethical assessment.
• Facilitates critical analysis of how the computing domain measures AI ethics.
• Supports the advancement of meaningful AI governance through improved assessment practices.