Improving preliminary clinical diagnosis accuracy through knowledge filtering techniques in consultation dialogues

Symptom descriptions by ordinary people are often inaccurate or vague when seeking medical advice, which often leads to inaccurate preliminary clinical diagnoses. To address this issue, we propose a deep learning model named the knowledgeable diagnostic transformer (KDT) for the natural language processing (NLP)-based preliminary clinical diagnoses.

The KDT extracts symptom-disease relation triples (h,r,t) from patient symptom descriptions by using a proposed bipartite medical knowledge graph (bMKG). To avoid too many relation triples causing the knowledge noise issue, we propose a knowledge inclusion-exclusion approach (KIA) to eliminate undesirable triples (a knowledge filtering layer). Next, we combine token embedding techniques with the transformer model to predict the diseases that patients may encounter.

To train the KDT, a medical diagnosis question-answering dataset (named MDQA dataset) containing large-scale, high-quality questions (patient syndrome description) and answering (diagnosis) corpora with 2.6M entries (1.07GB in size) in Mandarin was built. We also train the KDT with the National Institutes of Health (NIH) English dataset (MedQuAD). The KDT marks a transformative approach by achieving a remarkable accuracy of 99% for different evaluation metrics when compared with the baseline transformers used for the NLP-based preliminary clinical diagnoses approaches.

In essence, our study not only demonstrates the effectiveness of the KDT in enhancing diagnostic precision but also underscores its potential to revolutionize the field of preliminary clinical diagnoses. By harnessing the power of knowledge-based approaches and advanced NLP techniques, we have paved the way for more accurate and reliable diagnoses, ultimately benefiting both healthcare providers and patients. The KDT has the potential to significantly reduce misdiagnoses and improve patient outcomes, marking a pivotal advancement in the realm of medical diagnostics.