Reliability of COVID-19 symptom checkers as national triage tools: an international case comparison study

Triage is a critical component of the pandemic response. It affects morbidity, mortality and how effectively the available healthcare resources are used. In a number of nations the pandemic has sponsored the adoption of novel, online, patient-led triage systems-often referred to as COVID-19 symptom checkers. The current safety and reliability of these new automated triage systems remain unknown.

We tested six symptom checkers currently in use as triage tools at a national level against 52 cases simulating COVID-19 of various severities to determine if the symptom checkers appropriately triage time-critical cases onward to healthcare contact. We further analysed and compared each symptom checker to determine the discretionary aspects of triage decision-making that govern the automated advice generated.

Of the 52 clinical presentations, the absolute rate of onward referral to any form of healthcare contact was: Singapore 100%, the USA 67%, Wales 65%, England 62%, Scotland 54% and Northern Ireland 46%. Triage decisions were broadly based on either estimates of 'risk' or 'disease severity'. Risk-based symptom checkers were more reliable, with severity-based symptom checkers often triaging time-critical cases to stay home without clinical contact or follow-up.

The COVID-19 symptom checkers analysed here were unable to reliably discriminate between mild and severe COVID-19. Risk-based symptom checkers may hold some promise of contributing to pandemic case management, while severity-based symptom checkers-the CDC and NHS 111 versions-confer too much risk to both public and healthcare services to be deemed a viable option for COVID-19 triage.