Conversational artificial intelligence for geriatric screening: A mixed-methods feasibility study

Geriatric assessment (GA) can identify clinically relevant vulnerabilities in older adults, but remains underused because of time, staffing, and workflow barriers. Conversational artificial intelligence (AI) may provide a scalable approach to support geriatric screening within clinical practice.

We conducted a two-phase mixed-methods feasibility study evaluating Geriatric Risk Assessment and Capability Evaluation (GRACE), a conversational AI platform for selected structured Practical Geriatric Assessment (PGA)-based screening items, compared with manual PGA assessment. Seventy adults aged ≥65 years were recruited from an academic health system and affiliated community sites. Phase 1 (n = 46) informed platform development and refinement, and Phase 2 (n = 24) evaluated the refined platform. Screening performance was evaluated at the scored-item level using sensitivity, specificity, predictive values, Cohen's κ, and Gwet's AC1. Usability was measured with the System Usability Scale (SUS) and semi-structured interviews with participants and clinicians explored acceptability and implementation.

In Phase 1 (n = 46), GRACE demonstrated sensitivity of 0.68, specificity of 0.96, and negative predictive value (NPV) of 0.96. Following platform refinement, Phase 2 (n = 24) demonstrated sensitivity of 0.77, specificity of 0.97, positive predictive value of 0.72, NPV of 0.98, Cohen's κ of 0.72, and Gwet's AC1 of 0.94. Phase 2 represented the primary estimate of refined-platform performance; overall Phase 1-to-Phase 2 differences were numerical rather than statistically significant. Descriptively pooled across all 70 participants, sensitivity was 0.71, specificity 0.96, NPV 0.97, κ 0.65, and AC1 0.92. Performance varied across domains, with stronger performance in several structured domains and lower sensitivity in functional and psychosocial domains. Participants generally reported favorable usability, and patient and clinician interviews supported a potential role for GRACE in pre-visit information gathering and targeted clinician follow-up.

The refined GRACE platform demonstrated promising performance for structured geriatric screening, including high specificity and NPV, supporting its potential role in pre-visit screening and prioritization of clinician follow-up. GRACE should be used as a clinician-guided, human-in-the-loop screening tool rather than as a standalone method for excluding geriatric deficits because GRACE showed lower sensitivity in some domains and NPV depends on prevalence. Larger prospective studies are needed to validate the refined platform and evaluate its implementation and clinical impact.