Using mobile meditation app data to predict future app engagement: an observational study

Meditation with mobile apps has been shown to improve mental and physical health. However, regular, long-term meditation app use is needed to maintain these health benefits, and many people have a difficult time maintaining engagement with meditation apps over time. Our goal was to determine the length of the timeframe over which usage data must be collected before future app abandonment can be predicted accurately in order to better target additional behavioral support to those who are most likely to stop using the app.

Data were collected from a randomly drawn sample of 2600 new subscribers to a 1-year membership of the mobile app Calm, who started using the app between July and November of 2018. App usage data contained the duration and start time of all meditation sessions with the app over 365 days. We used these data to construct the following predictive model features: total daily sessions, total daily duration, and a measure of temporal similarity between consecutive days based on the dynamic time warping (DTW) distance measure. We then fit random forest models using increasingly longer periods of data after users subscribed to Calm to predict whether they performed any meditation sessions over 2-week intervals in the future. Model fit was assessed using the area under the receiver operator characteristic curve (AUC), and an exponential growth model was used to determine the minimal amount of data needed to reach an accurate prediction (95% of max AUC) of future engagement.

After first subscribing to Calm, 83.1% of the sample used the Calm app on at least 1 more day. However, by day 350 after subscribing, 58.0% of users abandoned their use of the app. For the persistent users, the average number of daily sessions was 0.33 (SD = 0.02), the average daily duration of meditating was 3.93 minutes (SD = 0.25), and the average DTW distance to the previous day was 1.50 (SD = 0.17). The exponential growth models revealed that an average of 64 days of observations after subscribing to Calm are needed to reach an accurate prediction of future app engagement.

Our results are consistent with existing estimates of the time required to develop a new habit. Additionally, this research demonstrates how to use app usage data to quickly and accurately predict the likelihood of users' future app abandonment. This research allows future researchers to better target just-in-time interventions towards users at risk of abandonment.