Related Experiment Video
Updated: Jul 4, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Exploratory Evaluation of Large Language Models for Reducing Language Bias in Systematic Review Screening
Junki Ikeguchi1, Hiroaki Ueshima2,3, Hiroshi Tamura1,2
1Graduate School of Informatics, Kyoto University, Kyoto, Japan.
Studies in Health Technology and Informatics
|July 3, 2026
Summary
Large language models (LLMs) can help systematic reviews overcome language bias. Direct multilingual processing by LLMs maintained high sensitivity for non-English studies, unlike translation methods.
Area of Science:
- Medical Informatics
- Natural Language Processing
- Systematic Reviews
Background:
- Systematic reviews often exclude non-English studies due to resource limitations, introducing language bias.
- Large language models (LLMs) offer potential solutions for processing multilingual data in research.
Purpose of the Study:
- To evaluate if direct multilingual processing by LLMs reduces language-based disparities in systematic review abstract screening compared to translation-mediated approaches.
Main Methods:
- Six state-of-the-art LLMs were tested on English and non-English abstracts.
- Performance was assessed using direct multilingual screening versus screening translated abstracts.
- Key metrics included sensitivity, specificity, F1 score, balanced accuracy, and workload reduction.
Main Results:
- All LLMs showed high sensitivity (≥0.938) on English abstracts.
- Translation-mediated screening significantly decreased sensitivity for some models (0.47-0.54).
- Direct multilingual LLM processing maintained higher sensitivity (0.71-1.00) for non-English abstracts, with notable model variations.
Conclusions:
- Direct multilingual LLM screening shows promise in reducing language-related sensitivity disparities in systematic reviews.
- Further research is needed to understand the impact on downstream meta-analytic bias.
