Medical vs. Organizational Complaints: A Machine Learning Analysis Reveals Divergent Patterns in Patient Reviews...

Background: The growth of digital patient feedback presents a new opportunity for healthcare quality monitoring. This study addresses the need to automatically classify the content of patient reviews to identify primary sources of dissatisfaction. Objective: The purpose of this study is to develop a machine learning algorithm for classifying negative patient reviews into two core categories: medical content (M-pertaining to diagnosis, treatment, and outcomes) and organizational support (O-pertaining to logistics, cost, and communication). We aim to identify which type of concern prevails and to analyze variations across cities, patient gender, and medical specialties. Methods: A database of 18,680 negative patient reviews (rated 1 star) was compiled from the Russian aggregator infodoctor.ru for the period from July 2012 to August 2023. A training set was created using an independent annotation procedure with three experts. A logistic regression model was trained to classify reviews into M and O categories, demonstrating an accuracy of 88.5%. Results: The analysis revealed a significant structural shift in Moscow, where since 2021, medical (M) complaints began to prevail over organizational (O) ones. This trend was not observed in St. Petersburg or other major Russian cities. Notably, in St. Petersburg, M-type reviews were more common within the most represented medical specialties, whereas O-type reviews consistently dominated in other cities. Gender differences were most pronounced in St. Petersburg, where women were more frequently authors of M reviews and men of O reviews. Conclusions: The developed algorithm provides a valuable tool for the automated monitoring of patient feedback. It enables healthcare managers to distinguish between clinical and service-related issues, facilitating targeted improvements in medical service quality and patient satisfaction.