Classifying Chinese Medicine Constitution Using Multimodal Deep-Learning Model.

Area of Science:

• Integrative Medicine
• Artificial Intelligence
• Biomedical Informatics

Background:

• Traditional Chinese Medicine (TCM) constitution classification is crucial for personalized healthcare.
• Current diagnostic methods often rely on subjective assessments.
• Objective and data-driven approaches are needed for TCM constitution analysis.

Purpose of the Study:

• To develop a multimodal deep-learning model for classifying balanced and unbalanced Chinese medicine constitutions.
• To integrate data from tongue and face images, pulse waves, and health information.
• To enhance the accuracy and objectivity of TCM constitution identification.

Main Methods:

• Collected multimodal data (tongue/face images, pulse waves, health info) from 540 subjects.
• Employed deep learning models (CNNs, RNNs, fully connected networks) for feature extraction.
• Constructed feature fusion and decision fusion models for integrated analysis.

Main Results:

• Optimal models identified: ResNet18 (images), Gate Recurrent Unit (pulse), entity embedding (health info).
• Feature fusion outperformed decision fusion.
• Multimodal data analysis demonstrated superior classification performance by compensating for single-mode information loss.

Conclusions:

• Multimodal data fusion effectively supplements single-source information, enhancing classification accuracy.
• Deep learning technology is effective for identifying body constitution in TCM.
• This research supports the modernization and intelligent application of Chinese medicine.