Using Artificial Intelligence for Text Screening in a Systematic Review of Cardiotoxicity

Artificial intelligence (AI), capable of analyzing vast volume of data rapidly, presents a promising solution to optimize literature screening for systematic reviews (SRs). Using the INSIDE (artificial INtelligence to Support Informed DEcision making) platform, we compared the performance of AI against the "gold standard" traditional SR method in the context of prostate cancer (PC) to assess whether AI could potentially improve efficiency and quality of screening.

Publications from traditional screening of four SRs (focused on PC therapies and potential cardiotoxicity) were compared with the AI-based approach. Publications were ranked based on relevance scores. Work saved over sampling (WSS), that is, efforts saved by automatically excluding nonrelevant publications, determined efficiency. For a quality analysis, data visualization using a scatter plot suggested the proportions of "relevant," "irrelevant," and "not screened" records.

For AI-based screening, an efficiency analysis used publications from the traditional approach (n = 3363) including 278 relevant records. Of the total 3363 publications, the first ranking method screened 2365 and active learning used 3361 records. This approach was more efficient; fewer publications were required to be screened to identify 80% and 95% of 278 relevant publications (WSS@80% 20.3%; WSS@95% 9.4%). Screening efficiency increased with active learning (WSS@80% 54.0%; WSS@95% 54.8%). A scatter plot analysis presented broader search results with the Dimensions database yielding 384 465 publications and helped identify outlier articles.

This study confirms the impact of an AI-based approach in optimizing the SR process. It highlights best practices and benchmarks to assess the efficiency and possibly quality of literature screening, supporting the integration of AI into future SRs.

Systematic reviews (SRs) help create a detailed and unbiased summary on a specific research question. This summary is based on published information. Development of SRs using the traditional method requires detailed in-person review of the records, which takes a lot of time and effort. With the use of artificial intelligence (AI), the key data from a large amount of text are identified faster. This process requires a review of fewer records to find the most relevant ones, which saves time. The aim of this study was to understand how an AI tool, known as INSIDE PC, could help with SRs. This study looked at how well INSIDE PC worked compared with the traditional method for SRs. The AI method scored articles based on their relevance with respect to the topic of this SR. Data visuals or graphs were used to compare data points and remove irrelevant records from the review. This process decreased workload and saved time. The AI method also used a learning algorithm known as active learning. This helps AI tools learn from a small training sample data. Useful records were identified much faster by this method, with less efforts. The results showed that AI could improve the ease and speed of reviewing records for SRs. It is important that these AI methods are tested and improved to meet the needs of SRs.