Comparing raw score difference, multilevel modeling, and structural equation modeling methods for estimating...

Dyadic data analysis is commonly used in psychological research involving pairs of individuals in a nested relationship, such as parent and child, student and teacher, and pairs of spouses. There are several methods for calculating dyadic discrepancy (i.e., difference) scores, and purpose of the present study was to explore which of these methods produced the most accurate discrepancy estimates and most accurate outcome prediction.

Using a Monte Carlo simulation, the present study compared three methods for estimating discrepancy scores in dyad pairs: raw score difference (RSD), empirical Bayes estimates from multilevel modeling (MLM), and factor scores from structural equation modeling (SEM). Design factors for this simulation included intraclass correlation (ICC), cluster number, reliability estimates, effect size of discrepancy, and effect size variance.

Results suggest discrepancy estimates from MLM had poor reliability compared to RSD and SEM methods. These findings were driven primarily by having a high ICC, high effect size variance, and low number of clusters. None of the design factors had an appreciable impact on either the RSD or SEM estimates.

RSD and SEM methods performed similarly, and are recommended for practical use in estimating discrepancy values. MLM is not recommended as it featured comparatively poor reliability.