Related Experiment Video
Updated: Jul 23, 2025
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
A Novel Mobile App to Identify Patients With Multimorbidity in the Emergency Setting: Development of an App and
Claire Barthlow Rosen1, Sanford Eugene Roberts1, Solomiya Syvyk1
1Hospital of the University of Pennsylvania, Philadelphia, PA, United States.
Background:
Multimorbidity is associated with an increased risk of poor surgical outcomes among older adults; however, identifying multimorbidity in the clinical setting can be a challenge.
Objective:
We created the Multimorbid Patient Identifier App (MMApp) to easily identify patients with multimorbidity identified by the presence of a Qualifying Comorbidity Set and tested its feasibility for use in future clinical research, validation, and eventually to guide clinical decision-making.
Methods:
We adapted the Qualifying Comorbidity Sets' claims-based definition of multimorbidity for clinical use through a modified Delphi approach and developed MMApp. A total of 10 residents input 5 hypothetical emergency general surgery patient scenarios, common among older adults, into the MMApp and examined MMApp test characteristics for a total of 50 trials. For MMApp, comorbidities selected for each scenario were recorded, along with the number of comorbidities correctly chosen, incorrectly chosen, and missed for each scenario. The sensitivity and specificity of identifying a patient as multimorbid using MMApp were calculated using composite data from all scenarios. To assess model feasibility, we compared the mean task completion by scenario to that of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator (ACS-NSQIP-SRC) using paired t tests. Usability and satisfaction with MMApp were assessed using an 18-item questionnaire administered immediately after completing all 5 scenarios.
Results:
There was no significant difference in the task completion time between the MMApp and the ACS-NSQIP-SRC for scenarios A (86.3 seconds vs 74.3 seconds, P=.85) or C (58.4 seconds vs 68.9 seconds,P=.064), MMapp took less time for scenarios B (76.1 seconds vs 87.4 seconds, P=.03) and E (20.7 seconds vs 73 seconds, P<.001), and more time for scenario D (78.8 seconds vs 58.5 seconds, P=.02). The MMApp identified multimorbidity with 96.7% (29/30) sensitivity and 95% (19/20) specificity. User feedback was positive regarding MMApp's usability, efficiency, and usefulness.
Conclusions:
The MMApp identified multimorbidity with high sensitivity and specificity and did not require significantly more time to complete than a commonly used web-based risk-stratification tool for most scenarios. Mean user times were well under 2 minutes. Feedback was overall positive from residents regarding the usability and usefulness of this app, even in the emergency general surgery setting. It would be feasible to use MMApp to identify patients with multimorbidity in the emergency general surgery setting for validation, research, and eventual clinical use. This type of mobile app could serve as a template for other research teams to create a tool to easily screen participants for potential enrollment.
Insights
The Multimorbid Patient Identifier App (MMApp) accurately identifies patients with multimorbidity, demonstrating high sensitivity and specificity. This mobile tool is feasible for clinical use and research, aiding in identifying older adults for surgical risk assessment.
Area of Science:
- Geriatric Surgery
- Health Informatics
- Clinical Decision Support
Background:
- Multimorbidity increases surgical risk in older adults.
- Clinical identification of multimorbidity presents challenges.
Purpose of the Study:
- Developed the Multimorbid Patient Identifier App (MMApp) to identify patients with multimorbidity.
- Tested MMApp feasibility for clinical research, validation, and decision-making.
Main Methods:
- Adapted multimorbidity definition for clinical use via modified Delphi approach.
- Ten residents evaluated MMApp with 5 hypothetical emergency general surgery scenarios (50 trials).
- Calculated MMApp sensitivity and specificity; compared task completion time with ACS-NSQIP-SRC.
Main Results:
- MMApp demonstrated 96.7% sensitivity and 95% specificity for identifying multimorbidity.
- Task completion time was comparable to the ACS-NSQIP-SRC, with MMApp being faster in most scenarios.
- User feedback highlighted MMApp's usability, efficiency, and usefulness.
Conclusions:
- MMApp accurately identifies multimorbidity with high sensitivity and specificity.
- Feasible for emergency general surgery settings for research and clinical application.
- Serves as a template for developing similar screening tools.
