[Comparison of generative artificial intelligence and physicians in differential diagnosis and treatment planning...

To compare the quality of differential diagnoses and diagnostic and treatment plans formulated by generative artificial intelligence (GenAI) and by specialist physicians practicing western medicine or traditional Chinese medicine at the stage of preparing the initial progress note for hospitalized patients.

A dual-path controlled comparative study was conducted using the same medical record data. Hospitalized patients admitted to the Department of Respiratory and Critical Care Medicine at Haihe Hospital, Tianjin University, from April to November 2025 were selected. After admission, standardized history taking, physical examination, and relevant ancillary examinations were performed by specialist physicians, and textual medical record data were compiled. According to the method used to formulate the diagnostic and treatment plans, two groups were established: a physician group, comprising a western medicine physician subgroup and a traditional Chinese medicine physician subgroup and a GenAI group, comprising a western medicine GenAI subgroup and a traditional Chinese medicine GenAI subgroup. In the physician group, diagnostic and treatment plans were independently formulated by specialist physicians. In the GenAI group, diagnostic and treatment recommendations were generated by investigators using a locally deployed, customized DeepSeek model service. Evaluation indices for differential diagnosis included diagnostic redundancy, diagnostic omission, diagnostic prioritization, and diagnostic appropriateness. Evaluation indices for diagnostic and treatment plans included plan completeness and plan precision. The differential diagnoses and diagnostic and treatment plans generated by the physician and GenAI groups were separately rated by experts in western medicine or traditional Chinese medicine using a 5-point scale.

A total of 96 medical records were included, including records from 28 patients with pulmonary infections and 9 patients with rare diseases. (1) Quality of differential diagnosis: For diagnostic omission, scores were higher in the western medicine GenAI subgroup than in the western medicine physician subgroup [score: 4.00 (4.00, 4.50) vs. 4.00 (3.75, 4.25), P<0.05], and scores were higher in the traditional Chinese medicine GenAI subgroup than in the traditional Chinese medicine physician subgroup [score: 5.00 (4.00, 5.00) vs. 3.00 (2.00, 3.00), P<0.05]. For diagnostic appropriateness, scores were higher in the western medicine GenAI subgroup than in the western medicine physician subgroup [score: 4.25 (4.00, 4.50) vs. 4.00 (4.00, 4.25), P<0.05], whereas no statistically significant difference was found between the traditional Chinese medicine GenAI and traditional Chinese medicine physician subgroups. For diagnostic redundancy and diagnostic prioritization, no statistically significant differences were found between the corresponding GenAI and physician subgroups in either western medicine or traditional Chinese medicine. (2) Quality of diagnostic and treatment plans: For plan completeness, scores were higher in the western medicine GenAI subgroup scored higher than in the western medicine physician subgroup [score: 4.25 (4.00, 4.50) vs. 4.00 (3.75, 4.25)], and were also higher in the traditional Chinese medicine GenAI subgroup than in the traditional Chinese medicine physician subgroup [score: 5.00 (4.00, 5.00) vs. 3.00 (2.00, 3.00); both P<0.05]. For plan precision, no statistically significant differences were found between the corresponding GenAI and physician subgroups in either western medicine or traditional Chinese medicine.

At the stage of preparing the initial progress note for hospitalized patients, GenAI may offer potential advantages in reducing diagnostic omissions and traditional Chinese medicine, as well as in improving diagnostic appropriateness in western medicine. GenAI may serve as an adjunctive tool to support clinical decision-making by specialist physicians.