Toward reliable machine learning with Congruity: a quality measure based on formal concept analysis

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Machine learning (ML) and deep learning (DL) models face transparency and trust issues in critical applications like healthcare.
• Lack of transparency in ML/DL models raises ethical concerns regarding their predictions and decisions.

Purpose of the Study:

• To introduce a novel measure, Congruity, to quantify the reliability of ML/DL model results.
• To demonstrate the correlation between Congruity and the established Accuracy metric for ML/DL models.

Main Methods:

• Congruity is defined using a lattice derived from formal concept analysis on training data.
• The measure assesses the similarity between incoming data items and the model's training dataset.
• Formal Concept Analysis (FCA) and lattice extraction are employed.

Main Results:

• Experimental results show a strong positive correlation between Congruity and ML model Accuracy.
• The correlation value between Congruity and Accuracy exceeds 80% across various ML models.
• Congruity effectively indicates the reliability of ML/DL model predictions.

Conclusions:

• Congruity offers a valuable tool for understanding and trusting ML/DL model outputs.
• The proposed measure enhances transparency by quantifying the relationship between training data and input data.
• This work contributes to building more reliable and trustworthy AI systems in sensitive domains.