New Recommendations on the Use of R-Squared Differences in Multilevel Model Comparisons

Area of Science:

• Statistics
• Multilevel Modeling
• Quantitative Psychology

Background:

• Researchers often use R-squared differences to evaluate model terms in multilevel models (MLMs).
• Existing methods for calculating R-squared differences in MLMs are confusing and have limitations.
• Previous studies provided incomplete or misleading recommendations for MLM R-squared measures.

Purpose of the Study:

• To clarify and resolve issues surrounding R-squared difference measures for comparing multilevel models.
• To provide a general set of R-squared difference measures applicable to various MLM comparisons.
• To offer concrete procedures for selecting appropriate R-squared measures in MLM analyses.

Main Methods:

• Defined a comprehensive set of total, within-cluster, and between-cluster R-squared difference measures.
• Developed step-by-step procedures for identifying relevant measures for specific MLM comparisons.
• Utilized simulated and analytic demonstrations to validate the proposed methods and measures.

Main Results:

• Identified and addressed limitations in previous MLM R-squared difference studies.
• Demonstrated how the general set of measures and procedures overcome prior methodological issues.
• Provided tools for automatic computation and visualization of R-squared difference measures.

Conclusions:

• The proposed framework offers a unified approach to R-squared differences in multilevel modeling.
• Recommendations are provided for practical application, including extensions for pseudo-R-squareds and model-building strategies.
• This work aims to improve the accurate assessment of effect sizes and term importance in MLMs.