The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
J Vrijsen1, C L van Erpecum2, S E de Rooij3
1University Medical Centre Groningen, Department of Epidemiology, University of Groningen, Hanzeplein 1, FA40, P.O. Box 30 001, 9700 RB, Groningen, The Netherlands. j.vrijsen@umcg.nl.
BMC Psychology
|April 29, 2021
Summary
A new digital Ruff Figural Fluency Test (RFFT) shows good criterion validity and is a feasible alternative for large-scale cognitive assessments. However, scores from the digital and paper-and-pencil RFFT are not interchangeable.
Area of Science:
- Cognitive Psychology
- Neuroscience
- Digital Health
Background:
- The Ruff Figural Fluency Test (RFFT) is a validated cognitive assessment but is time-consuming and labor-intensive.
- A digital version of the RFFT was developed for independent administration on an iPad with automatic score computation.
Purpose of the Study:
- To investigate the validity and reliability of the digital RFFT.
- To compare the digital RFFT with the traditional paper-and-pencil version.
Main Methods:
- A cross-over design was used, with participants randomly assigned to either the digital or paper-and-pencil RFFT first, followed by the other version.
- Test-retest reliability was assessed with a second digital RFFT administration one week later.
- Automatic and manual scoring (two raters) were compared for the digital RFFT, alongside comparisons between digital and paper-and-pencil RFFT using Intraclass Correlation Coefficients (ICC), sensitivity, specificity, and Bland-Altman plots.
Main Results:
- Automatic scoring of the digital RFFT demonstrated high agreement with manual scoring for unique designs (UD) and perseverative errors (PE), indicating excellent criterion validity.
- Agreement between the digital and paper-and-pencil RFFT was moderate for UD and poor for PE, with fewer UD observed on the digital version.
- Test-retest reliability for the digital RFFT was moderate (ICC = 0.74).
- Digital RFFT performance correlated with education level and younger age, supporting its ability to discriminate between groups.
Conclusions:
- The automatically scored digital RFFT exhibits good criterion and convergent validity.
- Moderate test-retest reliability and low agreement with the paper-and-pencil RFFT suggest potential learning effects.
- The digital RFFT is a valid and reliable tool for measuring executive cognitive function in large-scale studies, but its scores are not interchangeable with the traditional version.
