Applying Object Detection and Large Language Model to Establish a Smart Telemedicine Diagnosis System with Chatbot: A...

Background: The scarcity of medical resources and personnel has worsened due to COVID-19. Telemedicine faces challenges in assessing wounds without physical examination. Evaluating pressure injuries is time consuming, energy intensive, and inconsistent. Most of today's telemedicine platforms utilize graphical user interfaces with complex operational procedures and limited channels for information dissemination. The study aims to establish a smart telemedicine diagnosis system based on YOLOv7 and large language model. Methods: The YOLOv7 model is trained using a clinical data set, with data augmentation techniques employed to enhance the data set to identify six types of pressure injury images. The established system features a front-end interface that includes responsive web design and a chatbot with ChatGPT, and it is integrated with a database for personal information management. Results: This research provides a practical pressure injury staging classification model with an average F1 score of 0.9238. The system remotely provides real-time accurate diagnoses and prescriptions, guiding patients to seek various medical help levels based on symptom severity. Conclusions: This study establishes a smart telemedicine auxiliary diagnosis system based on the YOLOv7 model, which possesses capabilities for classification and real-time detection. During teleconsultations, it provides immediate and accurate diagnostic information and prescription recommendations and seeks various medical assistance based on the severity of symptoms. Through the setup of a chatbot with ChatGPT, different users can quickly achieve their respective objectives.