Self-reported binary gender prediction from personality traits: Alignment between machine learning importance and...

Area of Science:

• Psychology
• Machine Learning
• Personality Science

Background:

• Personality differences are often measured using effect sizes like Cohen's d and Mahalanobis distance.
• Machine learning models are increasingly used for personality classification, but their feature importance is not well-understood in relation to traditional measures.

Purpose of the Study:

• To investigate the relationship between machine learning model-based feature importance and classical measures of group separation in personality data.
• To compare how feature importance rankings from various machine learning models align with univariate effect sizes and multivariate discriminant structures across different personality inventories.

Main Methods:

• Trained multiple linear and nonlinear machine learning models (Logistic Regression, Random Forest, Extra Trees, LightGBM, Explainable Boosting Machine) to predict binary gender from personality trait scores.
• Derived feature importance rankings using SHAP and permutation importance, and compared them with univariate effect sizes (Cohen's d), linear discriminant weights, and Mahalanobis distance contributions.
• Utilized two large datasets: Big5 (18,062 participants) and 16PF (44,324 participants).

Main Results:

• Trait-importance rankings were highly stable across model-resampling configurations in both datasets.
• In the Big5 dataset, univariate effect size rankings closely matched model-based importance (rank correlations 0.98–1.00).
• In the 16PF dataset, alignment between univariate effect size and model-based importance was moderate (ρ=0.56–0.64), while alignment with multivariate discriminant structure remained strong (ρ=0.83–0.85).

Conclusions:

• Divergence between univariate and model-based importance reflects covariance-mediated multivariate weighting, not novel predictive structures.
• When trait intercorrelations are low, univariate summaries approximate multivariate importance.
• When trait intercorrelations are substantial, multivariate geometry influences trait contributions, and model-based explainability metrics align with discriminant structure.