Scalable deep text comprehension for Cancer surveillance on high-performance computing

Area of Science:

• Bioinformatics
• Computational Biology
• Machine Learning

Background:

• Deep Learning (DL) models are increasingly sophisticated and computationally demanding for bioinformatics.
• Large datasets and complex models challenge research feasibility.
• Data parallelism on High-Performance Computing (HPC) offers a solution by distributing workloads.

Purpose of the Study:

• To evaluate the scalability of training Deep Learning models using data parallelism.
• To optimize information extraction from cancer pathology reports via Convolutional Neural Networks (CNNs).
• To compare the performance of different optimizer functions and batch sizes.

Main Methods:

• Implemented Downpour Stochastic Gradient Descent for data parallelism.
• Trained a CNN on a massive dataset of cancer pathology reports.
• Utilized the Titan supercomputer at Oak Ridge Leadership Computing Facility for scalability experiments.
• Compared various worker node counts, batch sizes, and optimizer functions (Adadelta, RMSProp, Adam).

Main Results:

• Adadelta converged at lower validation loss but required more epochs than RMSProp.
• Adam optimizer achieved near-optimal validation loss significantly faster, converging in 4.5 epochs with batch sizes of 16 and 32.
• Scalability was evaluated across different numbers of worker nodes.

Conclusions:

• Networked training processes are scalable across multiple compute nodes using message passing interface.
• The implemented approach achieved higher classification accuracy than traditional machine learning algorithms.