Automated Summarization of Military Medical Records Using AI: Evaluation Via the OPTICA Framework

Military medical fitness evaluations require physicians to rapidly review extensive and heterogeneous medical records to determine service eligibility and duty limitations. This process is time-consuming, cognitively demanding, and often conducted under significant operational pressure, contributing to inefficiencies and physician burnout. Advances in artificial intelligence (AI), particularly large language models (LLMs), offer potential solutions for automating and streamlining medical record review. However, real-world evaluations of AI-based summarization tools in military clinical settings remain limited. This study describes the development, implementation, and pilot evaluation of an AI-based system designed to automatically summarize medical records to support military physicians during fitness evaluations.

This was a prospective implementation case study. We developed an AI-based medical record summarization system integrating Optical Character Recognition (OCR), document intelligence tools, a proprietary document splicing engine, and LLMs to process heterogeneous medical documents. The system was designed to extract, organize, and present clinically relevant information in a concise, structured summary tailored for medical fitness assessments. The system was deployed in a prospective observational pilot study conducted over 6 months within a military medical evaluation framework. More than 12,000 cases were processed. Summaries generated by the AI system were reviewed by military physicians as part of routine clinical workflows. Evaluation was guided by the OPTICA framework, focusing on clinical relevance, accuracy, completeness, usability, and potential impact on workflow efficiency. Physicians provided structured feedback via questionnaires.

Across the pilot period, the AI system successfully processed more than 12,000 reservist cases and generated structured summaries integrated into routine military clinical workflows. Iterative refinements focused on OCR confidence thresholds, hallucination mitigation, document segmentation, and clinician-guided prompt optimization. Clinicians reported that the summaries were generally clinically relevant and facilitated faster understanding of complex medical histories. The system reduced the need for manual document navigation and repetitive data extraction. Physicians indicated improved efficiency in medical record review and perceived cognitive load reduction. No direct patient care decisions were automated; the AI functioned strictly as a decision-support tool, with final judgments retained by clinicians.

An AI-based medical record summarization system can meaningfully support military medical fitness evaluations by improving efficiency, reducing documentation burden, and supporting clinician well-being. This pilot demonstrates the feasibility of integrating LLM-based summarization into real-world military medical workflows. This implementation case study suggests that OPTICA may serve not only as a procurement/deployment evaluation framework, but also as a practical roadmap for iterative clinical AI development and deployment in real-world healthcare settings. Further research is warranted to evaluate long-term clinical impact, scalability, and optimization of human-AI collaboration in high-stakes medical environments.