Search research articles

Search research articles

Related Experiment Video

Updated: Apr 20, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evaluating large language models for AI-assisted grading: a framework and case study in higher education.

Yago Saez¹, Luis Mario Garcia², Asuncion Mochon³

¹Computer Science Department, Universidad Carlos III de Madrid, Avda. de la Universidad 30, Madrid, 28911, Madrid, Spain. yago.saez@uc3m.es.

Scientific Reports

|April 18, 2026

Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Coin.AI: A Proof-of-Useful-Work Scheme for Blockchain-Based Distributed Deep Learning.

Entropy (Basel, Switzerland)·2020

Same author

Advances in Recent Nature-Inspired Algorithms for Neural Engineering.

Computational intelligence and neuroscience·2020

Same author

Association of Porcine Swine Leukocyte Antigen (SLA) Haplotypes with B- and T-Cell Immune Response to Foot-and-Mouth Disease Virus (FMDV) Peptides.

Vaccines·2020

Same author

Synthetic and real data sets for benchmarking non-cryptographic hash functions.

Data in brief·2019

Same author

A Comparison of Machine Learning and Deep Learning Techniques for Activity Recognition using Mobile Devices.

Sensors (Basel, Switzerland)·2019

Same author

Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments.

Sensors (Basel, Switzerland)·2018

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

This study evaluated large language models (LLMs) for grading student assignments. DeepSeek-R1 closely matched human instructors in grading accuracy and feedback quality for data analytics courses.

Area of Science:

Artificial Intelligence
Educational Technology
Data Science

Background:

Higher education institutions are exploring AI tools for administrative tasks.
Large language models (LLMs) show potential in automating educational assessments.
Evaluating LLM performance in grading requires robust methodologies.

Purpose of the Study:

To empirically evaluate six state-of-the-art LLMs for grading university-level student assignments.
To compare LLM-generated grades and feedback against human instructor evaluations.
To propose a transferable framework for benchmarking LLMs in higher education assessment.

Main Methods:

Selected six advanced LLMs for evaluation in a data analytics and machine learning course.
Developed a systematic process for prompt design and model selection.

Keywords:

AI-assisted grading Benchmarking Methodology Educational assessment Evaluation framework Feedback quality Higher education Instructional technology Large Language Models (LLMs)

Related Experiment Videos

Last Updated: Apr 20, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Utilized statistical and semantic measures to assess grading accuracy and feedback quality.

Included a cost analysis for AI-assisted grading.

Main Results:

DeepSeek-R1 demonstrated the highest alignment with human instructor evaluations in both grading accuracy and feedback quality.
The study identified key parameters for effective LLM benchmarking in educational contexts.
The proposed framework provides a replicable methodology for evaluating AI in assessment.

Conclusions:

LLMs can effectively assist in grading university assignments, with DeepSeek-R1 showing promising results.
A standardized framework is crucial for the systematic evaluation and adoption of AI in higher education.
This research offers a transferable methodology for educators and researchers to assess AI-assisted grading systems.