Increased interpretation of deep learning models using hierarchical cluster-based modelling.

Area of Science:

• Machine Learning
• Chemometrics
• Data Science

Background:

• Linear models struggle with data exhibiting inhomogeneity or non-linearities.
• Local modeling by data clustering can improve performance on locally linear data.
• Interpretability of complex deep learning models remains a challenge.

Purpose of the Study:

• To extend Hierarchical Cluster-based Partial Least Squares Regression (HC-PLSR) to deep learning architectures.
• To enhance the interpretability of deep learning models through local modeling.
• To evaluate the performance of new Hierarchical Cluster-based Deep Learning models (HC-CNNs, HC-RNNs, HC-SVRs).

Main Methods:

• Implementation of HC-CNNs, HC-RNNs, and HC-SVRs.
• Testing models on a simulated dataset with distinct non-linear relationships.
• Application to Fourier Transform Infrared (FT-IR) spectroscopic data for molecular weight prediction.

Main Results:

• HC-CNN, HC-RNN, and HC-SVR outperformed HC-PLSR on the simulated non-linear dataset.
• For FT-IR data, complex models offered minimal prediction gains over HC-PLSR.
• Significant differences in feature importance were observed across local models, aiding interpretability.

Conclusions:

• Local modeling in deep learning improves interpretability by revealing variations in feature importance.
• Hierarchical clustering combined with deep learning offers a powerful approach for analyzing complex datasets.
• The choice of model complexity should consider the specific characteristics of the dataset.