CT: A Controllable Transformer for Multi-Task TCM Facial Inspection.

Area of Science:

• Artificial Intelligence in Medicine
• Traditional Chinese Medicine Diagnostics
• Computer Vision

Background:

• Traditional Chinese Medicine (TCM) facial inspection offers a non-invasive health assessment but lacks standardization due to subjective practitioner judgment.
• Applying AI to TCM facial diagnosis presents challenges including spatial heterogeneity, complex task correlations, and integrating domain knowledge.

Purpose of the Study:

• To develop an AI model that addresses the limitations of subjective TCM facial inspection.
• To improve standardization and scalability in TCM diagnostics through artificial intelligence.

Main Methods:

• Proposed CT (Control Transformer), a hierarchical multi-task vision transformer.
• Incorporated task-specific spatial priors using a ControlNet-based module with 'Mingtang' partition masks.
• Utilized a Mixture-of-Experts (MoE) backbone for specialized knowledge sharing and a knowledge-graph-infused FiLM module for theory-guided feature modulation.

Main Results:

• CT significantly outperformed existing baseline models on a self-constructed TCM Facial Diagnosis (TCM-FD) dataset.
• Ablation studies and qualitative analyses confirmed the effectiveness and interpretability of individual model components.

Conclusions:

• The Control Transformer (CT) presents a practical and effective paradigm for AI-assisted TCM diagnostics.
• The developed methods demonstrate a way to integrate domain knowledge and handle complex correlations in medical AI applications.