Extending reliability to intensive longitudinal data with the Kalman filter.

Area of Science:

• Psychometrics
• Statistics
• Behavioral Science

Background:

• Reliability is a cornerstone of measurement in group-based, single-time-point analyses.
• This critical aspect of measurement is often neglected in the analysis of repeated observations.
• Traditionally, reliability has been conceptualized as a between-person attribute.

Purpose of the Study:

• To redevelop the concept of reliability for within-person, single-subject designs.
• To propose a novel coefficient, kappa (κ), for assessing reliability in single-subject time-series data.
• To extend the application of reliability assessment to time-dependent within-person variability.

Main Methods:

• Development of a new reliability coefficient, kappa (κ), for single-subject designs.
• Utilizing latent variable time series models, specifically state space models.
• Extension to multi-subject state space models accommodating continuous or discrete individual variation.
• Derivation and analytical proof of kappa (κ) and its relationship to existing reliability coefficients.

Main Results:

• Introduction of kappa (κ), a new coefficient for within-person reliability in single-subject designs.
• Demonstration that kappa (κ) applies the general definition of reliability (true score variance to total variance ratio) to time-dependent within-person variability.
• Establishment of the coefficient's foundation in state space modeling for time series data.

Conclusions:

• The proposed kappa (κ) coefficient provides a robust measure of reliability for within-person, time-dependent data.
• This work extends psychometric principles to single-subject research, addressing a gap in reliability assessment for repeated measures.
• The findings offer a new tool for researchers analyzing complex within-person processes over time.