Related Experiment Video
Updated: Nov 5, 2025

Mixed Reality for Education MRE Implementation and Results in Online Classes for Engineering
Published on: June 23, 2023
Validation Evidence from using Generalizability Theory in a Basic-Science Course: Reliability of Course-Grades from
Michael J Peeters1, M Kenneth Cor2, Sai Hs Boddu1,3
1University of Toledo College of Pharmacy & Pharmaceutical Sciences, Toledo, OH.
Combining multiple exams using Generalizability Theory (G-Theory) significantly enhances the reliability of course grades for high-stakes decisions. More exams mean fewer items are needed overall for acceptable reliability.
Area of Science:
- Educational Measurement
- Psychometrics
- Pharmaceutics Education
Background:
- Reliability is crucial for high-stakes decision-making in education.
- Individual exams may lack sufficient reliability for these decisions.
- Combining multiple assessments can potentially improve overall reliability.
Purpose of the Study:
- To investigate the utility of Generalizability Theory (G-Theory) in improving the reliability of course grades.
- To determine how combining multiple exams affects composite reliability.
- To explore the relationship between the number of exams, items per exam, and overall reliability.
Main Methods:
- Utilized Generalizability Theory (G-Theory) and G-String IV software to combine reliability data from multiple exams.
- Conducted decision studies to analyze changes in course-grade reliability based on the number of exams and items.
- Applied G-Theory to data from 101 first-year PharmD students across three exams in a pharmaceutics course.
Main Results:
- The composite reliability of course grades, after combining three exams using G-Theory, was 0.71, surpassing the reliability of individual exams (KR-20 range: 0.65-0.69).
- Increased numbers of exam occasions required fewer items per exam and fewer total items to achieve acceptable composite reliability.
- Different combinations of multiple-choice questions (MCQs) per exam and the number of exam occasions can yield acceptable reliability.
Conclusions:
- G-Theory provides critical validation evidence for high-stakes decision-making by enhancing the reliability of combined course exams.
- While combining exams improved course-grade reliability, further improvements are possible.
- Employing more exam occasions allows educators to reduce the number of items needed per exam and in total, optimizing assessment development.
More Related Videos
10:17Improving Student Outcomes with an Adaptable Molecular Cloning Course-Based Undergraduate Research Experience
Published on: November 15, 2024
07:32Use of Galvanic Skin Responses, Salivary Biomarkers, and Self-reports to Assess Undergraduate Student Performance During a Laboratory Exam Activity
Published on: February 10, 2016
Related Concept Videos
Reliability and Validity
Comparing Experimental Results: Student's t-Test
Random and Systematic Errors
Data Validation
Key parameters for method validation include:
Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
One-Way ANOVA: Equal Sample Sizes
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...