The use of bibliography enriched features for automatic citation screening

Area of Science:

• Information Science
• Computer Science
• Biomedical Informatics

Background:

• Systematic reviews rely on citation screening, a process often supported by text mining (TM) to save time and effort.
• Class imbalance in search results is a significant challenge for TM models in automatic citation screening.
• Standard TM models using only titles and abstracts often yield suboptimal performance.

Purpose of the Study:

• To investigate the impact of incorporating full bibliography data alongside titles and abstracts on text classification performance for automatic citation screening.
• To evaluate the effectiveness of different feature representations and classification models when using enriched citation data.

Main Methods:

• Experimentation with binary and Word2vec feature representations using Support Vector Machine (SVM) models.
• Comparison of three model types (binary-non-linear, Word2vec-linear, Word2vec-non-linear kernels) across 4 software engineering and 15 medical review datasets.
• Evaluation of performance using metrics such as recall, work saved over sampling (WSS), and Matthews correlation coefficient (MCC).

Main Results:

• Bibliography-enriched data demonstrated consistent performance improvements in recall, WSS, and MCC for three of four large software engineering datasets.
• Performance on medical datasets showed varied results, but was generally the same or better in most cases.
• The inclusion of bibliography data showed a positive trend in enhancing classification performance.

Conclusions:

• Incorporating bibliography data into text mining models offers potential for improving automatic citation screening performance.
• While promising, the results indicate that the benefits of bibliography data inclusion are not yet conclusive across all dataset types.
• Further research may be needed to fully leverage bibliography data for optimizing systematic review processes.