Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Multiple Comparison Tests01:13

Multiple Comparison Tests

4.5K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.5K
Feedback Inhibition00:46

Feedback Inhibition

57.3K
Biochemical reactions are occurring constantly in cells, converting starting substances to different products, usually with the help of enzymes that speed the reactions. Without enzymes, it would take far too long for most reactions to occur to be useful to the cell!
57.3K
Feedback Loops01:01

Feedback Loops

64.7K
In most cases, excessive hormone production is prevented by negative feedback—a loop that starts with a stimulus inducing the release of a particular substance, like a hormone, to maintain a certain level before triggering a signal that results in a decrease in further release of the hormone.
64.7K
Distance Corrections01:15

Distance Corrections

301
To achieve precise distance measurements, especially in surveying and construction, certain corrections must be applied to account for potential sources of error like the standardization errors, temperature variations, and slope adjustments.Standardization error emerges when measurement equipment undergoes changes, such as wear, repairs, or weather impacts. To address this, surveyors compare the equipment’s readings to a standard. This process identifies any deviation that might lead to...
301
Power Factor Correction01:20

Power Factor Correction

557
The power transmission to a factory involves the transfer of apparent power, a combination of active and reactive power. The power factor measures how effectively electrical power is converted into useful work output. The ratio of the real power (KW) that does the work to the apparent power (KVA) supplied to the circuit.
557
Mate Choice01:20

Mate Choice

11.8K
Mate choice—the decision about whom to mate with—is a type of natural selection, since animals must reproduce to pass down their genes. Mate choice is also called intersexual selection because the behavior occurs between the sexes.
11.8K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

RiTeK: A Dataset for Large Language Models Complex Reasoning over Textual Knowledge Graphs in Medicine.

Findings of ACL. ACL·2026
Same author

RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models.

Proceedings of the conference. Association for Computational Linguistics. Meeting·2026
Same author

MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skills.

Proceedings of the conference. Association for Computational Linguistics. European Chapter. Conference·2026
Same author

ChatCLIDS: Simulating Persuasive AI Dialogues to Promote Closed-Loop Insulin Adoption in Type 1 Diabetes Care.

Proceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence·2026
Same author

Social Determinants of Health and 1-Year Buprenorphine Initiation Among Justice-Involved Veterans.

JAMA network open·2026
Same author

From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations.

Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing·2026

Related Experiment Video

Updated: Feb 13, 2026

The Joint Effect of Social Comparison and Social Distance on Evaluation of Intertemporal Choice Outcomes in Event-related Potential Studies
08:24

The Joint Effect of Social Comparison and Social Distance on Evaluation of Intertemporal Choice Outcomes in Event-related Potential Studies

Published on: August 25, 2023

1.2K

MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and

Zonghai Yao1, Aditya Parashar1, Huixue Zhou2

  • 1University of Massachusetts, Amherst.

Proceedings of the Conference. Association for Computational Linguistics. North American Chapter. Meeting
|February 12, 2026
PubMed
Summary

This study introduces MCQG-SRefine, a novel framework using large language models (LLMs) to generate high-quality multiple-choice questions (MCQG) for medical licensing exams. The method improves question quality and difficulty assessment.

More Related Videos

A Two-interval Forced-choice Task for Multisensory Comparisons
07:13

A Two-interval Forced-choice Task for Multisensory Comparisons

Published on: November 9, 2018

11.5K
Control of Eating Behavior Using a Novel Feedback System
04:48

Control of Eating Behavior Using a Novel Feedback System

Published on: May 8, 2018

11.7K

Related Experiment Videos

Last Updated: Feb 13, 2026

The Joint Effect of Social Comparison and Social Distance on Evaluation of Intertemporal Choice Outcomes in Event-related Potential Studies
08:24

The Joint Effect of Social Comparison and Social Distance on Evaluation of Intertemporal Choice Outcomes in Event-related Potential Studies

Published on: August 25, 2023

1.2K
A Two-interval Forced-choice Task for Multisensory Comparisons
07:13

A Two-interval Forced-choice Task for Multisensory Comparisons

Published on: November 9, 2018

11.5K
Control of Eating Behavior Using a Novel Feedback System
04:48

Control of Eating Behavior Using a Novel Feedback System

Published on: May 8, 2018

11.7K

Area of Science:

  • Artificial Intelligence
  • Natural Language Processing
  • Medical Education

Background:

  • Automatic question generation (QG) is crucial for AI and NLP applications like intelligent tutoring and dialogue systems.
  • Generating high-quality multiple-choice questions (MCQG) for professional exams, such as the United States Medical Licensing Examination (USMLE), is challenging due to domain expertise and reasoning requirements.
  • Current large language models (LLMs) face limitations in professional MCQG, including outdated knowledge, hallucinations, and prompt sensitivity, leading to suboptimal question quality and difficulty.

Purpose of the Study:

  • To develop an LLM self-refine-based framework (MCQG-SRefine) for generating high-quality USMLE-style questions from medical cases.
  • To enhance the quality and difficulty of generated questions through expert-driven prompt engineering and iterative self-critique and self-correction.
  • To introduce an automated assessment metric using LLM-as-Judge to replace costly expert evaluations.

Main Methods:

  • Proposed the MCQG-SRefine framework, integrating LLM self-critique and correction mechanisms.
  • Employed expert-driven prompt engineering to guide the question generation process.
  • Developed an LLM-as-Judge metric for automatic evaluation of question quality and difficulty.

Main Results:

  • MCQG-SRefine significantly improved human expert satisfaction with the quality and difficulty of generated USMLE-style questions.
  • The framework effectively converts medical cases into challenging and relevant MCQs.
  • The LLM-as-Judge metric demonstrated reliable and expert-aligned assessments, offering an alternative to manual expert review.

Conclusions:

  • MCQG-SRefine offers a robust solution for generating high-quality medical MCQG, addressing limitations of current LLMs.
  • The self-refinement approach enhances question relevance and difficulty, crucial for professional exams.
  • Automated assessment using LLM-as-Judge provides a scalable and cost-effective method for evaluating generated questions.