A note on the expected value of the Rand index

Area of Science:

• Statistics
• Data Analysis
• Psychometrics

Background:

• The adjusted Rand index (ARI) is a common metric for comparing clustering results.
• Accurate expectation calculation is crucial for statistical inference and hypothesis testing with ARI.
• Existing approximations may not fully capture the true behavior of the ARI.

Purpose of the Study:

• To compare the exact expectation of the adjusted Rand index (ARI) under the hypergeometric distribution with an approximation under the multinomial distribution.
• To evaluate the accuracy of the multinomial approximation, particularly its behavior with varying sample sizes.
• To assess the impact of using different ARI expectations within a hypothesis testing framework.

Main Methods:

• Theoretical comparison of ARI expectations derived from hypergeometric and multinomial distributions.
• Mathematical proofs for minimum and maximum differences between the two expectations.
• Simulation studies to demonstrate the practical implications of using each expectation.

Main Results:

• The multinomial approximation provides a poor estimate of the ARI's exact expectation.
• The discrepancy between the two expectations can grow with increasing sample size.
• Simulations show significant differences in ARI values depending on the expectation used.
• The multinomial approximation leads to an over-rejection of the null hypothesis in testing scenarios.

Conclusions:

• The multinomial approximation for ARI expectation is unreliable and should be used with caution.
• Researchers should be aware of the potential for inflated Type I errors when using this approximation in hypothesis testing.
• The exact hypergeometric expectation is recommended for accurate ARI calculation and inference.