大型视觉语言模型在舌头图像描述生成和物理结构推理在传统中医学的应用 (TongueVLM):模型开发和验证研究研究
Chengdong Peng1,2, Jun Gao1, Nuo Yang2,3
1School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, Anhui, China.
JMIR medical informatics
|March 12, 2026
概括
一个新的TongueVLM模型可以解释传统中医 (TCM) 诊断的舌头图像. 这种人工智能系统有助于识别身体结构和病理信息,增强TCM诊断能力.
科学领域:
- 人工智能的人工智能
- 医疗成像医学成像
- 传统中国医药 传统中国医药
背景情况:
- 传统中医 (TCM) 诊断依赖于解释舌头图像来评估身体结构.
- 模拟TCM从业人员对舌头图像分析的推理对于开发智能诊断系统至关重要.
研究的目的:
- 开发和验证用于TCM舌头图像分析的专门AI模型.
- 为模型赋予TCM特定的理解和推理能力.
主要方法:
- 一个多模式的大型模型,TongueVLM,设计了视觉编码器,模态融合和语言解码器模块.
- 使用CLIP-ViT进行视觉特征提取,并使用LLaMA对7B参数TCM-domain语言模型进行微调.
- 在模态融合模块中使用剩余架构来实现视觉和TCM术语之间的概念对齐.
主要成果:
- 在三个评估任务中,TongueVLM与一般的大型模型相比,表现优越.
- 在多式联络测试数据集上,准确率分别为79.8%,78.6%和60.7%.
- 在精度方面表现优于LLaVA-OneVision和Qwen2.5-VL-7B,文本生成速度约为24令牌/秒.
结论:
- 舌头VLM模型有效地生成舌头图像描述和关于TCM物理构成的理由.
- TongueVLM非常适合集成到传统中医药的智能诊断系统中.
相关概念视频
00:49The Tongue and Taste Buds
41.9K
The surface of the tongue is covered with various small bumps called papillae, which either distribute what has been ingested (filiform papillae) or contain the sensory taste (or gustatory) receptor cells (fungiform, circumvallate, and foliate papillae). Embedded within each taste-related papilla are the taste buds—clusters of 30 to 100 gustatory receptor cells.
41.9K
01:26Assessment of the Mouth
1.4K
A thorough mouth assessment, including inspection and palpation of the lips, gums, tongue, tonsils, uvula, and pharynx, is crucial in detecting potential health issues. Diseases ranging from oral cancer to systemic conditions like diabetes could be identified early through careful oral examination. This article provides a detailed guide on conducting a comprehensive mouth assessment.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
1.4K