Related Experiment Video
Updated: Aug 5, 2026

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits
Published on: September 27, 2019
Mechanisms of Effect Size Differences Between Researcher Developed and Independently Developed Outcomes: An
Joshua B Gilbert1, James Soland2
1Center for School Behavioral Health, Massachusetts General Hospital, Boston, Massachusetts; United States.
Abstract:
Differences in effect sizes between researcher-developed (RD) and independently developed (ID) outcome measures are widely documented but poorly understood in education research. We conduct a meta-analysis using item-level outcome data to test potential mechanisms that explain differences in effects by RD or ID outcome type. Our analysis of 54 effect sizes from 32 studies shows that greater standard deviations of item-specific treatment effects predict larger effect sizes and reduce the observed difference between RD and ID measures from 0.14 SDs to 0.12 SDs, leaving the remainder attributable to other factors. The findings advance our understanding of how item properties predict educational intervention outcomes and underscore the importance of considering item-level data for building theory in education research.
Related Concept Videos
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Group Design
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
Methods of Medium Optimization
Comparing Experimental Results: Student's t-Test
Regression Toward the Mean