Accurate fatty liver disease diagnosis with a multi-source feature fusion model on the segmented tongue image dataset

More than 100 million individuals in rural areas of China are suffered from Fatty Liver Disease (FLD). However, health clinics in remote regions often lack the necessary professional expertise and expensive ultrasound equipment for regular liver disease screening. Delayed treatment frequently leads to liver cirrhosis and cancer, imposing substantial economic burden on both public health systems and affected families.

Traditional Chinese Medicine emphasizes the strong association between tongue characteristics and liver health. Leveraging machine learning to model the relationship between tongue images and FLD can enable rapid, non-invasive, large-scale screening in medically underserved areas. However, existing studies in this domain often rely on small-scale private datasets, which can result in unverifiable model performance. Moreover, most studies have employed generic convolutional neural networks for feature extraction, causing a lack of interpretability. The goal of our research is to address above-mentioned questions.

In this study, we first introduced a Multi-source Feature Fusion-based Tongue Diagnosis Framework for FLD diagnosis (MFF-TDF). In addition, we developed and released a standardized tongue image dataset with physiological indicators and FLD annotations, comprising 5,717 samples, which to our knowledge is the largest public dataset in this domain. Finally, we evaluated the effectiveness of the proposed method through extensive experiments and enhanced model interpretability using shapley additive explanations and counterfactual analysis.

When conducting fusion modeling with tongue images and some basic physiological indicators (such as sex, age, height, etc.), FLD's prediction performance in the population reached F1-score 0.797, Recall 0.847, and AUC 0.924. This performance significantly exceeds that of the state-of-the-art methods published in this domain.

This study developed an automated and explainable method for tongue diagnosis that facilitated the low-cost, speedy screening of FLD in large-scale populations, and contributed the largest public dataset to support future modeling research in this field.