Measuring and testing the agreement of matrices

Area of Science:

• Psychometrics and Quantitative Psychology
• Computational Statistics
• Matrix Analysis

Background:

• Comparing agreement between two n x n matrices is crucial in experimental psychology.
• Common agreement indices rely on element-wise products.
• Less common, but useful, indices use within-row and within-column gradients.

Purpose of the Study:

• To provide a computational toolkit for matrix agreement analysis.
• To implement both established and novel agreement indices.
• To offer robust statistical testing for matrix agreement.

Main Methods:

• Development of a MATLAB program suite.
• Implementation of agreement indices based on element-wise products.
• Implementation of agreement indices based on matrix gradients.
• Integration of matrix permutation tests for statistical significance.
• Provision of exact p-value calculations for small matrices.
• Development of resampling methods for approximate p-values in large matrices.

Main Results:

• A comprehensive suite of MATLAB programs for computing matrix agreement indices is available.
• The programs facilitate statistical testing of agreement using permutation tests.
• Methods for calculating exact and approximate p-values are included.
• The toolkit supports analysis for both small and large matrices.

Conclusions:

• The developed MATLAB programs offer a versatile tool for researchers in experimental psychology.
• The suite enables rigorous comparison of matrix agreement using various indices and statistical tests.
• This work facilitates the application of gradient-based agreement measures and robust permutation testing.