Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data

Area of Science:

• Artificial Intelligence in Medicine
• Deep Learning for Healthcare
• Clinical Data Analysis

Background:

• Transformer models have revolutionized natural language processing.
• Adapting Transformer architectures to complex longitudinal clinical data presents unique challenges.
• Existing methods may not fully capture the nuances of clinical data, including numerical values.

Purpose of the Study:

• To design a novel Transformer-based deep neural network (DNN) architecture for clinical data.
• To develop a model capable of jointly learning from longitudinal and non-longitudinal clinical data.
• To address the complexities of clinical data, such as numerical values associated with medical concepts.

Main Methods:

• Introduced the Hybrid Value-Aware Transformer (HVAT), a new DNN architecture.
• HVAT uniquely learns from numerical values of clinical codes (e.g., lab results).
• Employed a flexible longitudinal data representation termed 'clinical tokens'.

Main Results:

• A prototype HVAT model was trained on a case-control dataset.
• Achieved high performance in predicting Alzheimer's disease and related dementias.
• Demonstrated the model's potential for various clinical data-learning tasks.

Conclusions:

• The Hybrid Value-Aware Transformer (HVAT) is effective for analyzing complex clinical data.
• HVAT's ability to integrate numerical values and flexible data representation enhances predictive power.
• This deep learning approach shows significant potential for advancing clinical informatics and disease prediction.