Neural Networks as Entropic Systems: Applications in Digital Pathology.

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Machine learning interpretability

Background:

• Deep learning models in digital pathology are often considered "black boxes," hindering clinical adoption due to a lack of interpretability.
• Understanding the internal learning dynamics of neural networks is crucial for building trust and ensuring reliable clinical application.

Purpose of the Study:

• To develop and apply a framework for empirically characterizing the training-time learning dynamics of neural networks in digital pathology.
• To investigate the evolution of activation structure, weight trajectories, and spectral organization during model optimization.

Main Methods:

• Utilized TCGA BRCA whole-slide images with methylation proxies as regression targets.
• Trained a Vision Transformer model and meticulously tracked its intra-epoch behavior across 20 epochs.
• Measured activation structure, weight evolution, and spectral organization to analyze learning dynamics.

Main Results:

• Observed reproducible structural signatures during neural network training, including the formation of stable activation modules with increasing modularity and decreasing representation entropy (up to 60%).
• Weight trajectories demonstrated bounded diffusion, converging towards a stable regime, indicative of a damped stochastic process.
• Model attention shifted from stromal to nuclear regions, correlating with histologic indicators of replicative stress.

Conclusions:

• Neural networks exhibit predictable and quantifiable internal structure development during training, offering a mechanistic lens for interpretation.
• The framework provides a practical approach to visualize and measure learning dynamics, enhancing the interpretability of pathology AI models.
• Characterizing learning dynamics through entropy, modularity, and stochastic stabilization can demystify how AI models acquire biologically relevant representations.