Lessons from the TREC Plain Language Adaptation of Biomedical Abstracts (PLABA) track.

Area of Science:

• Biomedical informatics
• Natural Language Processing
• Health Communication

Background:

• Advances in Large Language Models (LLMs) offer potential for translating complex biomedical literature into patient-friendly language.
• The high-stakes nature of medical information necessitates thorough evaluation of LLM adaptations to ensure patient safety and understanding.

Purpose of the Study:

• To stimulate research in adapting biomedical literature to plain language using LLMs.
• To conduct a high-quality evaluation of systems performing plain language adaptation of biomedical abstracts.

Main Methods:

• The Plain Language Adaptation of Biomedical Abstracts (PLABA) track at Text Retrieval Conferences (TRC) 2023-2024.
• Task 1: Complete sentence-level rewriting of abstracts. Task 2: Identifying and replacing difficult terms.
• Automatic evaluation using professionally-written references and extensive manual evaluation by biomedical experts.

Main Results:

• Top models achieved human-level factual accuracy and completeness in rewriting abstracts but lacked simplicity and brevity.
• Automatic metrics showed poor correlation with manual judgments for abstract rewriting.
• Systems struggled with identifying and classifying difficult terms but excelled in generating accurate, complete, and simple replacements for Task 2.

Conclusions:

• The PLABA track demonstrates the potential of LLMs for making biomedical literature accessible to the public.
• Deficiencies in current LLM systems and automatic benchmarking tools for this task were identified.
• Further research is needed to improve LLM performance and develop better evaluation metrics for plain language adaptation.