A hybrid approach to large-scale systematic literature reviews: combining automated tools with text-mining techniques

Semi-automated tools used during the preliminary screening of articles in systematic reviews can start with a small set of seed articles and actively learn from human decisions to prioritise more relevant articles for subsequent screening. However, given that these tools are vulnerable to biases and lack clear stopping criteria, their performance in large-scale systematic reviews remains uncertain, especially in reviews covering broad subject areas that require a substantial number of representative seed articles. This article presents a hybrid approach that uses text-mining techniques combined with a semi-automated tool to effectively reduce, screen, and validate a large cohort of articles (N = 90,871).

A preliminary evaluation using simulations indicated that this approach has the potential to craft a comprehensive collection of seed articles that covers broad subject areas for semi-automated tools in a large-scale systematic review. The strengths and limitations of using a semi-automated tool alone in such a context are discussed. Our approach increases the efficiency of automated tools by providing a larger and more focused selection of articles to start with, optimising the learning process for those tools and reducing biases. Additionally, our approach could increase the transparency and reusability of keywords for future review updates.