ChatGPT Answers the 110-Question Laboratory Enzymology Student Exam: Pass or Fail?

Area of Science:

• Biochemistry
• Medical Education
• Artificial Intelligence

Background:

• Recent advancements in AI, particularly chatbots like ChatGPT, have spurred interest in their application across various fields.
• Concerns regarding the reliability and accuracy of AI-generated responses necessitate rigorous evaluation, especially in specialized scientific domains.
• Laboratory enzymology presents a complex area where AI knowledge assessment is crucial for educational integration.

Purpose of the Study:

• To assess the knowledge of ChatGPT-3.5 and ChatGPT-4.0 in laboratory enzymology.
• To compare the performance of these AI models against human students in the same subject.
• To explore the potential utility of ChatGPT in higher and medical education for enzymology.

Main Methods:

• An examination comprising 110 questions across four key topics in laboratory enzymology was administered.
• The accuracy of responses was evaluated for 52 students and both ChatGPT-3.5 and ChatGPT-4.0.
• A comparative analysis was conducted between student and AI performance, with a 60% overall pass rate observed.

Main Results:

• Students achieved an average score of 85.46%, significantly outperforming ChatGPT-3.5 (52.73%) and ChatGPT-4.0 (74.55%).
• ChatGPT-4.0 demonstrated superior performance compared to ChatGPT-3.5.
• Both AI versions excelled in questions related to enzymes in metabolism but struggled with laboratory analysis of enzymes.

Conclusions:

• ChatGPT demonstrated average performance in laboratory enzymology, scoring below human students.
• AI chatbots show potential as supplementary learning tools in higher and medical education, contingent upon significant optimization.
• Current AI capabilities in this specialized field do not yet permit the replacement of human educators or comprehensive learning resources.