Assessing the robustness of clinical trials by estimating Jadad's score using artificial intelligence approaches

Clinical trials are essential in medical science and are currently the most robust strategy for evaluating the effectiveness of a treatment. However, some of these studies are less reliable than others due to flaws in their design. Assessing the robustness of a clinical trial can be a very complex and time-consuming task, with factors such as randomization, masking and the description of withdrawals needing to be considered.

We built a program based on artificial intelligence (AI) approaches, designed to assess the robustness of a clinical trial by estimating its Jadad's score. The program is composed of five Recursive Neural Networks (RNN), each of them trained to spot one specific item constituting the Jadad's scale. After training, the algorithm was tested on two different validation sets (one from the original database: 35% of this database was used for validation and 65% for training; one composed of 10 articles, out of the original database, for which the Jadad's score has been computed by each contributor of this study).

After training, the algorithm achieved a mean accuracy of 96,2% (ranging from 93% to 98%) and a mean area under the curve (AUC) of 96% (ranging from 95% to 97%) on the first validation dataset. These results indicate good feature detection capacity for each of the five RNN. On the second validation dataset the algorithm extracted 100% of the item to retrieve for 70% of the articles and between 66% and 75% for 30% of the articles. Overall 85% of the items present in the second validation dataset were correctly extracted. None of the extracted items was misclassified.

We developed a program that can automatically estimate the Jadad's score of a clinical trial with a good accuracy. Automating the assessment of this metric could be very useful in a systematic review of the literature and will probably save clinicians time.