Automated Assessment of Argumentation Skills in Chemistry-Related Socioscientific Issues Using AI Chatbot

Area of Science:

• Science Education
• Chemistry Education
• Artificial Intelligence in Education

Background:

• Socioscientific issues (SSI) demand strong argumentation for informed decision-making.
• Manual assessment of argumentation quality using Toulmin's Argument Pattern (TAP) is laborious and subjective.
• There is a need for efficient, objective, and reliable methods to evaluate argumentation skills.

Purpose of the Study:

• To provide a guide for developing automated assessments of argumentation skills.
• Focuses on chemistry-related socioscientific issues (SSI).
• To leverage Generative Pre-trained Transformer (GPT) models for objective evaluation.

Main Methods:

• Development of an automated assessment tool using Claude, a GPT model.
• The guide facilitates the creation of these automated assessments.
• Utilizes a provided free package for implementation.

Main Results:

• A fully functional application for evaluating arguments was successfully produced by Claude.
• The automated assessment offers efficiency, objectivity, and reliability compared to manual methods.
• The developed guide is practical for educators and researchers.

Conclusions:

• Automated assessment using AI (Claude) is a viable solution for evaluating argumentation in science education.
• This approach enhances the assessment of chemistry-related SSI argumentation skills.
• The methodology presented is accessible and can be readily adopted.