Assessing reproducibility of data obtained with instruments based on continuous measurements.

Area of Science:

• Gerontology
• Biostatistics
• Research Methodology

Background:

• Reproducibility, or reliability, is crucial for research tool data.
• Intraclass correlation coefficients (ICC), Pearson correlations, and paired t tests are commonly used for reliability assessment in aging research.
• These methods are often incorrectly considered interchangeable, potentially leading to flawed instrument selection.

Purpose of the Study:

• To compare reliability estimates from ICC, Pearson correlations, and paired t tests.
• To determine the most preferable technique for assessing reliability in aging research.

Main Methods:

• A hypothetical dataset with 20 observations per participant was generated.
• Three scenarios were simulated: good agreement, systematic bias, and substantial random measurement error.
• Reliability estimates were calculated using ICC, Pearson correlations, and paired t tests for each scenario.

Main Results:

• All methods indicated reliability in the 'good agreement' scenario.
• In the 'systematic bias' scenario, ICC and t tests showed low reliability, while Pearson correlation indicated high reliability.
• In the 'random measurement error' scenario, ICC and Pearson accurately reflected low reliability, but the t test suggested high reliability.

Conclusions:

• Intraclass correlation coefficients (ICC) are the preferred technique for measuring reliability in aging research.
• While ICC has limitations, they are manageable, making it superior to Pearson correlations and paired t tests for reliability assessment.