Evaluation of the Finnish Exposure Notification App "Koronavilkku" According to Key Performance Indicators:...

Mobile exposure notification apps (ENAs) can play a significant role in future pandemics. Identifying lessons learned and areas for improvement from such apps, particularly within specific social and cultural contexts, is therefore crucial.

This study aimed to evaluate the Finnish ENA "Koronavilkku," according to chosen key performance indicators to assess the ENA's use and performance during the COVID-19 pandemic in Finland, and to identify strengths and improvement areas.

Using available data on Finland's ENA, we defined metrics for four key performance indicators: ENA use extent, enablers and barriers to use, the ENA's influence on user behavior, and overall acceptability. We performed an evaluation study combining system data and data from a survey of 4061 respondents conducted in April 2022. A sentiment analysis was performed on open-ended responses.

Koronavilkku ENA peak coverage reached approximately 53% of active smartphone users shortly after launch. Use declined over time, particularly among individuals diagnosed with COVID-19. User engagement remained high-among ENA users who received a token, the proportion that entered their token into the ENA was more than 50% during the whole study period. Males, individuals older than 34 years, and those living alone were less likely to use the ENA, while those with higher education and households with children were more likely to be ENA users. Almost 70% (64,642/94,280) of SARS-CoV-2-positive users reported not receiving a token from health authorities. Users reported higher adherence to pandemic guidance than nonusers, with prevalence ratios ranging from 1.24 (95% CI 1.19-1.29) to 1.65 (95% CI 1.45-1.91). Descriptive analyses suggested higher reported adherence among partially compliant users after exposure notifications, whereas little change was observed among noncompliant individuals. Overall acceptability was relatively high, with 68% of respondents using it at least once. However, 49% of those users eventually discontinued use. Our sentiment analysis of open-ended feedback suggests that users expressed more positive attitudes toward the ENA compared to nonusers. However, a substantial portion of the feedback reflected neutral or negative sentiments.

Based on observed ENA use trends, we recommend maintaining long-term public interest and engagement in ENAs during prolonged emergencies. Tailored communication should be developed encouraging uptake among demographic groups with lower adoption rates. Initial studies during app rollout could inform these strategies. A major barrier to the ENA's performance was token issuance; earlier and more systematic automation of token distribution in Finland could have enabled more timely exposure notifications and reduced bottlenecks. Future ENAs should define evaluation criteria during development, with periodic assessments measuring effectiveness and informing improvements. A deeper analysis of open-ended feedback provided for the Koronavilkku ENA using advanced language models could provide additional insights into user perceptions and concerns. By addressing these areas, future digital contact tracing tools may improve uptake, acceptability, and integration into pandemic response efforts.