Robustness is Not Dimensionality: On the Sensitivity of Component Comparability Coefficients to Sample Size

Area of Science:

• Psychometrics
• Statistical Analysis
• Personality Research

Background:

• Everett (1983) suggested replicability could determine the number of dimensions in component analysis.
• However, replicability is logically influenced by both sample size and dimensionality.
• The relationship between sample size, sample composition, and principal component replicability requires systematic examination.

Purpose of the Study:

• To investigate the effects of sample size and sample composition on the replicability of principal components.
• To assess the validity of using replicability as a sole criterion for determining dimensionality in component analysis.

Main Methods:

• Utilized observer ratings of personality from the California Adult Q-Set.
• Conducted 192 series of principal components analyses.
• Examined comparability coefficients to assess component replicability across different conditions.

Main Results:

• When sample size exceeds 20 subjects per item, the most comparable solution often includes as many components as items.
• Threshold decision rules (.90 or .85) show a substantial link between the number of components and sample size, excluding full solutions.
• Alternative criteria, like the Minimum Average Partial (MAP) rule, do not exhibit the same dependency on sample size.

Conclusions:

• Dimensionality cannot be reliably inferred from component robustness alone.
• Replicability and dimensionality are empirically and logically distinct.
• The influence of sample size on component retention criteria necessitates careful consideration in principal component analysis.