Bigger Is Not Always Better: Computational Efficiency in Lexical Prehospital Triage Modeling

In future large-scale combat operations, military medics will likely face staggering numbers of casualties in the face of resource constrained evacuation and forward medical care. Artificial intelligence (AI) may enable triage at scale and improve operational efficiencies. Although many assume that sophisticated large language models (LLMs) hold the most promise, simpler machine learning models (MLMs) may offer advantages in edge implementation because of lower computational burden. Here we search for computational efficiency-achieving equivalent clinical performance with computationally simpler AI-in the domain of on-scene triage.

We conducted a retrospective study via a civilian critical care air transport service from 2012 to 2021 (approved by the University of Pittsburgh IRB). Free-text impressions recorded by treating clinicians and records of patient lifesaving intervention (LSI; e.g., airway management; blood transfusion) were abstracted from the electronic health record. Two modeling approaches were used to predict LSI receipt from medic impression text. MLMs involved using an ensemble classifier model fed with word frequencies. LLMs involved using 2 transformer models (DistilBERT and BioBERT) fed with tokenized word embeddings.

A total of 12,913 patients were included (mean age = 52.3 years, 63% men). MLMs and LLMs produced broadly equivalent performance in predicting LSI (MLM: AUROC = .793 [.776, .810]; AP = .670 [.643, .695] vs. BioBERT: AUROC = .803 [.796, .816]; AP = .674 [.649, .698]) vs. DistilBERT: AUROC = .803 [.785, .811]; AP = .673 [.651, .688]). Broad predictive equipoise held across 7 independent LSI categories. Yet LLMs required considerably greater computational resources: Model training took 9.42 seconds for MLMs, 3 minutes 57 seconds for DistilBERT, and 7 minutes 14 seconds for BioBERT, whereas model prediction took .048 seconds for MLMs, 7.37 seconds for DistilBERT, and 13.36 seconds for BioBERT.

Our findings counter the "bigger is better" paradigm in medical AI: MLMs performed equivalently to LLMs in task mirroring on-scene triage while requiring a fraction of the computational resources. Nimble MLMs offer several advantages for military AI, including seamless deployment on edge computing devices and explainable decision support for combat medics. As AI technologies expand across warfighting domains, considerations of computational efficiency are critical to adopt and field new medical capabilities in the forward environment.