An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction

Area of Science:

• Natural Language Processing
• Machine Learning in Education

Background:

• Automatic Essay Score (AES) prediction systems are crucial for educational applications.
• AES systems analyze textual and grammatical features for score prediction.
• Linear regressions and classifiers require learning patterns to enhance scoring accuracy.

Purpose of the Study:

• To address catastrophic forgetting and reduce computational complexity in AES classifiers.
• To enhance prediction accuracy by resolving the forgetting problem.
• To propose an optimized Bi-directional Encoder Representation from Transformation (BERT) model.

Main Methods:

• An optimized BERT model, termed ABC-BERT-FTM, was developed by integrating the Artificial Bee Colony (ABC) algorithm.
• The ABC algorithm optimizes network parameters to mitigate the forgetting problem.
• The model was fine-tuned for improved performance.

Main Results:

• The optimized BERT model achieved a high prediction accuracy of up to 98.5% on ASAP and ETS datasets.
• The ABC algorithm effectively reduced the catastrophic forgetting problem in AES prediction.
• The proposed method demonstrated superior performance compared to existing approaches.

Conclusions:

• Optimizing BERT with meta-heuristic algorithms like ABC can resolve forgetting issues in AES systems.
• The ABC-BERT-FTM approach significantly increases AES prediction accuracy.
• This research offers a robust solution for automated essay scoring.