Model fit and measurement outcome in attachment measurements: a simulation study

Area of Science:

• Biostatistics
• Data Analysis
• Measurement Science

Background:

• Accurate modeling of biological or physical system changes over time is crucial.
• Distinguishing between rapid, short-term changes (burst model) and gradual, continuous changes (linear model) presents analytical challenges.
• Measurement error can obscure the true underlying pattern of change.

Purpose of the Study:

• To compare the goodness-of-fit of burst and linear models for simulated attachment level data.
• To evaluate how well different models capture true underlying change patterns under varying conditions of change magnitude and measurement error.
• To assess the reliability of least squares criteria for model selection in time-series data.

Main Methods:

• Simulated serial attachment level measurements using two distinct models: a burst model and a linear model.
• Incorporated normally distributed measurement error into the simulated data.
• Applied statistical criteria, including least squares, to determine which model best fit the simulated measurement series across different magnitudes of change.

Main Results:

• The burst model demonstrated a better fit than the linear model for a significant proportion of series generated by the burst model, regardless of change magnitude.
• For series generated by a linear model, the burst model was favored when changes were less than four times the standard deviation of measurement error.
• Model fit assessment using least squares may not reliably indicate the validity of the chosen model, as estimates can be biased depending on the true underlying process.

Conclusions:

• The choice of model (burst vs. linear) significantly impacts the estimation of change in serial measurements.
• Under certain conditions, particularly with smaller changes relative to measurement error, a burst model may appear to fit linear change data well, leading to potential misinterpretation.
• Relying solely on goodness-of-fit metrics like least squares can be insufficient for validating the accuracy of the underlying change model.