Problems of reliability and validity with similarity derived from category fluency

Area of Science:

• Cognitive Psychology
• Neuroscience
• Psycholinguistics

Background:

• Category fluency tasks are used to assess semantic memory.
• Exemplar similarity is a key metric derived from these tasks.
• The reliability and validity of derived similarity measures require thorough investigation.

Purpose of the Study:

• To evaluate the test-retest reliability of exemplar similarity derived from category fluency tasks.
• To assess the validity of derived similarities against directly rated similarities.
• To explore the implications of derived similarity unreliability for clinical interpretations.

Main Methods:

• 21 healthy participants completed a category fluency task twice, one week apart.
• Participants also rated the similarity of exemplar pairs.
• Similarity was derived from fluency data and compared to direct ratings using correlation and multidimensional scaling.

Main Results:

• Derived exemplar similarities were not stable over the one-week interval.
• There was low correspondence between derived similarities and directly rated similarities.
• Multidimensional scaling revealed differences between fluency task sessions and rating tasks.

Conclusions:

• Exemplar similarity derived from category fluency tasks exhibits poor temporal reliability.
• The validity of these derived similarities is questionable when compared to direct ratings.
• Observed differences in clinical populations may stem from task unreliability rather than true semantic impairments.