Related Experiment Video
Updated: Jun 9, 2026
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
Published on: April 14, 2023
A Multimodal Deep Learning Model for Objective Identification of Sub-health Status in Traditional Chinese Medicine
Diankun Cui1, Yue Meng2, Jinlian Liu3
1School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, 100029, China.
Introduction:
Sub-health is an intermediate state between health and disease. Early diagnosis in clinical practice is difficult, which delays timely intervention. Traditional Chinese Medicine (TCM) constitution theory divides sub-health into eight categories; however, existing diagnostic tools mainly focus on psychological assessment and lack objective phenotypic indicators. Multimodal technology provides a new direction for the objective diagnosis of sub-health by integrating objective human phenotypic images with expert experience.
Methods:
Tongue images, facial images, and infrared thermal images were collected from 1389 participants. In the pre-experiment, Model-1 consisted of a ResNet-18 network, an extended binary classification strategy, and a cross-attention fusion mechanism. Model-2 was developed based on Model-1 by integrating expert experience via Qwen2.5 using structural prompting. Accuracy, recall, precision, specificity, and AUC were used to evaluate model performance.
Results:
The overall performance of Model-1 was good, with most core indicators of body constitution exceeding 0.7. However, PDC (accuracy 0.722, AUC 0.682) and QDC (accuracy 0.732, AUC 0.705) showed relatively poor performance, characterized by high recall but low specificity. The core indicators of Model-2 improved further. Notably, PDC (accuracy 0.772, AUC 0.732) and QDC (accuracy 0.778, AUC 0.756) were significantly improved, and the imbalance between recall and specificity was also alleviated.
Discussion:
Multimodal fusion is critical for improving classification performance in TCM constitution recognition. The reasonable integration of clinical expert experience effectively optimizes feature fusion efficacy, highlighting the indispensable value of empirical expertise. Rooted in the empirical characteristics of TCM, embedding expert knowledge into the data-driven framework further enhances the robustness and credibility of the screening model.
Conclusion:
This study established an expert experience-enhanced multimodal model for subhealth and TCM constitution identification. The model achieves objective quantification and accurate classification of sub-health states, providing a valuable technical reference for population health management and early sub-health risk screening.
