Unbiased machine learning-assisted approach for conditional discretization of human performances

Area of Science:

• Statistics
• Machine Learning
• Human Performance Evaluation

Background:

• Performance discretization maps numerical data to ordinal categories.
• Norm-referenced discretization is crucial for evaluations like academic grading and salary increases.
• Existing Z-score methods only partially address conditional discretization.

Purpose of the Study:

• To develop novel methods for fully conditionally norm-referenced performance discretization.
• To introduce a multi-modal technique integrating machine learning and heuristics.
• To ensure conditional unbiasedness in performance ranking labels.

Main Methods:

• Proposed four novel approaches for conditional norm-referenced performance discretization.
• Employed a multi-modal technique combining unsupervised machine learning algorithms and a heuristic method.
• Developed a novel decision function to ensure conditional unbiasedness.

Main Results:

• Machine-learning-based methods showed superiority, achieving conditional unbiasedness degrees from 0.11 to 0.82.
• The heuristic method excelled in a specific dataset, reaching a conditional unbiasedness degree of 0.76.
• The multi-modal approach effectively leverages constituent methods for improved discretization.

Conclusions:

• The proposed multi-modal approach offers an effective solution for conditionally norm-referenced performance discretization.
• Novel machine learning and heuristic methods enhance conditional unbiasedness in performance rankings.
• This work advances performance evaluation by addressing limitations of existing Z-score methods.