Evaluation of DIABNET, a decision support system for therapy planning in gestational diabetes

DIABNET is a knowledge-based system designed to aid doctors with therapy planning in gestational diabetes. The system core is a qualitative model, implemented by a Causal Probabilistic Network, that is able to detect the insulin effectiveness on a daily basis. DIABNET analyses monitoring data and proposes quantitative changes in insulin therapy and qualitative diet modifications. This paper proposes an evaluation methodology to assess the system performance when working in a real scenario. The methodology manages the absence of a gold standard and includes: a subjective analysis based on questionnaires and an objective analysis based on a quantitative comparison of the system's and experts' proposals. The paper also shows the results of two experiments in which expert diabetologists evaluated the therapeutical advice provided by DIABNET during the follow up of 9 patients with gestational diabetes. DIABNET detected the need of a therapy modification in 92% of the cases showing its appropriateness for automatic alarm generation. Around 80% of the proposals were accepted by experts. The evaluation results are encouraging and allow characterisation of the system's performance when proposing therapy modifications. Evaluation in its turn helps to refine the knowledge managed by DIABNET and enables us to look towards the further clinical use of DIABNET as a decision tool in gestational diabetes integrated in a telemedicine service.