Scientific figures interpreted by ChatGPT: strengths in plot recognition and limits in color perception

Area of Science:

• Bioinformatics
• Computational Biology
• Artificial Intelligence in Science

Background:

• Large language models (LLMs) are increasingly explored for bioinformatics data analysis.
• The integration of image input capabilities (e.g., GPT-4V) in LLMs presents new opportunities for scientific figure interpretation.

Purpose of the Study:

• To evaluate the efficacy of ChatGPT (GPT-4V) in deciphering scientific figures within bioinformatics, particularly in cancer research.
• To assess the model's ability to explain plot types, apply biological knowledge, and generate figure legends.

Main Methods:

• Utilized ChatGPT (GPT-4V) to analyze diverse bioinformatics figures from cancer research domains.
• Included examples from sequencing data analysis, drug repositioning, and tumor clonal evolution studies.
• Assessed the model's performance in explaining figures, drafting legends, and summarizing findings.

Main Results:

• ChatGPT demonstrated proficiency in explaining various plot types and integrating biological context for enriched interpretations.
• The model exhibited limitations in accurately interpreting figures involving color perception and quantitative visual data.
• ChatGPT could generate draft figure legends and summaries, but required rigorous proofreading for accuracy and reliability.

Conclusions:

• LLM-based chatbots like ChatGPT (GPT-4V) show promise for assisting in the interpretation of bioinformatics figures.
• Current limitations in visual and quantitative analysis necessitate caution and human oversight for reliable scientific insights.
• Further development is needed to enhance the accuracy and robustness of AI tools for complex bioinformatics figure interpretation.