Assessing the performance of multimodal large language models in experimental information extraction from...

Advances in experimental techniques have expanded the volume of biological data. This has increased the demand for structured information extraction from papers, with large language models (LLMs) considered promising. However, challenges remain, including limited validation in biology and unclear applicability to multimodal tasks that integrate text with domain-specific figures, such as microscopic images and scatter plots. Here, we developed a multimodal LLM (MLLM)-based workflow to extract the experimental conditions and phase status from the text and figures of experimental papers on liquid-liquid phase separation and validated the effect of various inputs, prompts, and MLLM types. As a result, the MLLM-based extraction methods achieved F1-scores over 0.80 by processing each figure as a processing unit and inputting domain-specific prompts reflecting manual extraction guidance. This study demonstrates the potential and limitations of MLLMs for extracting experimental information using a focused set of LLPS papers as a model case and provides insights into the possibility of advancing multimodal approaches in biology.