An LLM-based hybrid approach for enhanced automated essay scoring

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Educational Technology

Background:

• Traditional Automated Essay Scoring (AES) systems rely on shallow lexical features (e.g., word frequency, sentence length).
• These methods often overlook critical aspects of text structure and semantics, leading to inadequate assessments of essay coherence and overall quality.
• There is a need for more sophisticated AES approaches that capture deeper linguistic nuances.

Purpose of the Study:

• To propose and evaluate a hybrid approach for Automated Essay Scoring (AES) that integrates multiple features from various linguistic levels.
• To address the limitations of shallow feature-based AES by incorporating structural and semantic information.
• To develop a more accurate and effective tool for evaluating student writing.

Main Methods:

• Developed a hybrid AES model combining shallow lexical features with deeper linguistic features (structure, semantics).
• Utilized a large language model (LLM) as the core of the hybrid approach.
• Experimented on standard essay datasets to validate the model's performance.

Main Results:

• The proposed hybrid LLM-based AES model significantly outperformed traditional shallow feature-based methods.
• The model also surpassed the performance of pure neural network models in essay scoring.
• Demonstrated the effectiveness of integrating diverse linguistic features for improved coherence and quality assessment.

Conclusions:

• A hybrid approach leveraging multiple linguistic levels, powered by large language models, offers a superior method for Automated Essay Scoring.
• This research advances the development of accurate and effective tools for automated assessment of student writing.
• Integrating semantic and structural features is crucial for overcoming the limitations of existing AES systems.