Geometry- and Accuracy-Preserving Random Forest Proximities.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Random forests are powerful classification and regression algorithms known for high predictive accuracy with minimal tuning.
• Pairwise proximities derived from random forests measure data point similarity, aiding tasks like variable importance and outlier detection.
• Current random forest proximity measures do not fully represent the learned data geometry.

Purpose of the Study:

• Introduce a novel definition of random forest proximities, termed Random Forest-Geometry- and Accuracy-Preserving proximities (RF-GAP).
• To develop a proximity measure that accurately reflects the underlying data geometry learned by random forests.
• To demonstrate the superiority of RF-GAP over traditional methods in various data analysis applications.

Main Methods:

• Developed a new definition for random forest proximities: RF-GAP.
• Proved that RF-GAP weighted sums (regression) or majority votes (classification) precisely match out-of-bag predictions.
• Empirically evaluated RF-GAP performance against traditional random forest proximities.

Main Results:

• RF-GAP exactly matches out-of-bag random forest predictions, preserving learned data geometry.
• Empirical results show RF-GAP outperforms traditional proximities in data imputation.
• RF-GAP provides enhanced outlier detection and data visualization consistent with learned data geometry.

Conclusions:

• RF-GAP offers a theoretically sound and empirically validated improvement over existing random forest proximity measures.
• The proposed RF-GAP method accurately captures data geometry, leading to better performance in downstream tasks.
• This novel proximity definition enhances the utility of random forests for complex data analysis and interpretation.