Information extraction from SMS text related to a reminder service for outpatients

This work evaluates the users' satisfaction with an SMS-based reminder system that is being used since about six years by an Italian healthcare organization. The system was implemented for reducing dropouts. This goal has been achieved, as dropout decreased from 8% to 4%. During these years, a number of reminded citizens, even not required, sent an SMS message back, with comments about the service, further requirements, etc. We collected some thousands of them. Their analysis may represent a useful feedback to the healthcare organization. We used conditional random fields as the information extraction method for classifying messages into appreciation, critique, inappropriateness, etc. The classification system achieved a very good overall performance (F1-measure of 94%), thus it can be used from here on to monitor the users' satisfaction in time.