Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Apr 20, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Evaluating large language models for AI-assisted grading: a framework and case study in higher education.

Yago Saez1, Luis Mario Garcia2, Asuncion Mochon3

  • 1Computer Science Department, Universidad Carlos III de Madrid, Avda. de la Universidad 30, Madrid, 28911, Madrid, Spain. yago.saez@uc3m.es.

Scientific Reports
|April 18, 2026
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Coin.AI: A Proof-of-Useful-Work Scheme for Blockchain-Based Distributed Deep Learning.

Entropy (Basel, Switzerland)·2020
Same author

Advances in Recent Nature-Inspired Algorithms for Neural Engineering.

Computational intelligence and neuroscience·2020
Same author

Association of Porcine Swine Leukocyte Antigen (SLA) Haplotypes with B- and T-Cell Immune Response to Foot-and-Mouth Disease Virus (FMDV) Peptides.

Vaccines·2020
Same author

Synthetic and real data sets for benchmarking non-cryptographic hash functions.

Data in brief·2019
Same author

A Comparison of Machine Learning and Deep Learning Techniques for Activity Recognition using Mobile Devices.

Sensors (Basel, Switzerland)·2019
Same author

Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments.

Sensors (Basel, Switzerland)·2018

This study evaluated large language models (LLMs) for grading student assignments. DeepSeek-R1 closely matched human instructors in grading accuracy and feedback quality for data analytics courses.

Area of Science:

  • Artificial Intelligence
  • Educational Technology
  • Data Science

Background:

  • Higher education institutions are exploring AI tools for administrative tasks.
  • Large language models (LLMs) show potential in automating educational assessments.
  • Evaluating LLM performance in grading requires robust methodologies.

Purpose of the Study:

  • To empirically evaluate six state-of-the-art LLMs for grading university-level student assignments.
  • To compare LLM-generated grades and feedback against human instructor evaluations.
  • To propose a transferable framework for benchmarking LLMs in higher education assessment.

Main Methods:

  • Selected six advanced LLMs for evaluation in a data analytics and machine learning course.
  • Developed a systematic process for prompt design and model selection.
Keywords:
AI-assisted gradingBenchmarking MethodologyEducational assessmentEvaluation frameworkFeedback qualityHigher educationInstructional technologyLarge Language Models (LLMs)

Related Experiment Videos

Last Updated: Apr 20, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K
  • Utilized statistical and semantic measures to assess grading accuracy and feedback quality.
  • Included a cost analysis for AI-assisted grading.

    • Main Results:

    • DeepSeek-R1 demonstrated the highest alignment with human instructor evaluations in both grading accuracy and feedback quality.
    • The study identified key parameters for effective LLM benchmarking in educational contexts.
    • The proposed framework provides a replicable methodology for evaluating AI in assessment.

    Conclusions:

    • LLMs can effectively assist in grading university assignments, with DeepSeek-R1 showing promising results.
    • A standardized framework is crucial for the systematic evaluation and adoption of AI in higher education.
    • This research offers a transferable methodology for educators and researchers to assess AI-assisted grading systems.