Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jun 7, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

SAS-bench: A fine-grained benchmark for evaluating short answer scoring with large language models.

Peichao Lai1, Kexuan Zhang2, Yi Lin3

  • 1School of Computer Science, Peking University, No.5 Yiheyuan Road, Haidian District, Beijing, 100871, China.

Neural Networks : the Official Journal of the International Neural Network Society
|June 5, 2026
PubMed
Summary

Related Concept Videos

Statistical Analysis System (SAS)01:14

Statistical Analysis System (SAS)

SAS, short for Statistical Analysis System, is a powerful data analysis, management, and visualization tool. Developed by the SAS Institute in the early 1970s, SAS has evolved into a comprehensive software suite used across various industries for statistical analysis, business intelligence, and predictive modeling.
Applications: SAS finds applications in numerous fields, including healthcare for clinical trial analysis, finance for risk assessment, marketing for customer data analysis, and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Degradation characteristics and mechanisms of ammonium and phenol by the direct ammonia oxidation bacterium Alcaligenes faecalis AP3-14.

Journal of hazardous materials·2026
Same author

Association of GLP-1RA discontinuation and risk of depressive and anxiety disorders in people with type 2 diabetes: a cohort study.

Nature metabolism·2026
Same author

Optical coherence tomography-derived macrophage arc as a novel biomarker for predicting adverse cardiovascular events in coronary artery disease: a multicentre study.

European heart journal. Imaging methods and practice·2026
Same author

Low-substrate nitrogen drives functional succession toward a cooperative Candidatus Brocadia consortium in anammox systems.

Bioresource technology·2026
Same author

Rethinking Token-Wise Feature Caching: Accelerating Diffusion Transformers With Dual Feature Caching.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2026
Same author

Integrated metagenomics unravels the microbial mechanisms driving greenhouse gas and odor emissions during composting.

Bioresource technology·2026
This summary is machine-generated.

We introduce SAS-Bench, a new benchmark for evaluating large language models (LLMs) in short answer scoring (SAS). It offers detailed, expert-annotated evaluations to improve automated grading accuracy and transparency in educational assessments.

Area of Science:

  • Artificial Intelligence
  • Natural Language Processing
  • Educational Technology

Background:

  • Automated subjective answer grading (Short Answer Scoring - SAS) is crucial for education and assessments.
  • Current SAS methods often lack detailed reasoning and transparency.
  • Large Language Models (LLMs) show promise but face bias and inconsistency issues.

Purpose of the Study:

  • To introduce SAS-Bench, a benchmark for LLM-based SAS tasks.
  • To enable fine-grained, step-wise scoring and detailed evaluation of model reasoning.
  • To address limitations of existing SAS approaches and LLM evaluators.

Main Methods:

  • Developed SAS-Bench with fine-grained scoring and expert-annotated error categories.
  • Created an open-source dataset of 1030 questions and 4109 student responses.
Keywords:
Automated assessmentLLM-as-a-judgeShort answer scoring

Related Experiment Videos

Last Updated: Jun 7, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

  • Conducted experiments using various LLMs and few-shot prompting techniques.

    • Main Results:

    • SAS-Bench facilitates detailed evaluation of LLM reasoning and explainability.
    • Identified significant challenges in scoring science-related questions.
    • Few-shot prompting demonstrated effectiveness in enhancing scoring accuracy.

    Conclusions:

    • SAS-Bench provides a robust framework for evaluating LLM-based SAS systems.
    • The findings offer insights for developing more accurate, fair, and transparent automated grading tools.
    • Future work can leverage SAS-Bench to advance LLM applications in education.