Foundation Models for Wearable Movement Data in Mental Health Research

Area of Science:

• Artificial Intelligence
• Biomedical Engineering
• Digital Health

Background:

• Foundation models and transformers have advanced AI, but health data modeling requires specialized adaptations.
• Wearable movement data, common in smartwatches, is relevant for clinical and mental health research due to its sequential nature.
• Existing AI models are not optimized for the unique characteristics of time-series wearable movement data.

Purpose of the Study:

• Introduce the Pretrained Actigraphy Transformer (PAT), the first open-source foundation model for time-series wearable movement data.
• Adapt transformer architectures and novel techniques for effective analysis of actigraphy data.
• Demonstrate PAT's capability in mental health prediction tasks.

Main Methods:

• Developed PAT using transformer-based architectures and patch embeddings.
• Pretrained PAT on a large dataset of 29,307 participants from a national U.S. sample.
• Evaluated PAT's performance on various mental health prediction tasks.

Main Results:

• PAT achieved state-of-the-art performance in multiple mental health prediction tasks.
• The model demonstrated effectiveness in analyzing sequential wearable movement data.
• PAT proved to be lightweight and easily interpretable.

Conclusions:

• PAT represents a significant advancement in applying foundation models to wearable sensor data for mental health research.
• The open-source nature of PAT facilitates broader adoption and further research in digital health.
• PAT offers a robust, interpretable, and high-performing tool for leveraging actigraphy data in mental health.