Numerical Analysis of Consensus Measures within Groups

Measuring the consensus for a group of ordinal-type responses is of practical importance in decision making. Many consensus measures appear in the literature, but they sometimes provide inconsistent results. Therefore, it is crucial to compare these consensus measures, and analyze their relationships. In this study, we targeted five consensus measures: Φ e (from entropy), Φ 1 (from absolute deviation), Φ 2 (from variance), Φ 3 (from skewness), and Φ m v (from conditional probability). We generated 316,251 probability distributions, and analyzed the relationships among their consensus values. Our results showed that Φ 1 ,   Φ e ,   Φ 2 , and Φ 3 tended to provide consistent results, and the ordering Φ 1 ≤ Φ e ≤ Φ 2 ≤ Φ 3 held at a high probability. Although Φ m v had a positive correlation with Φ 1 ,   Φ e ,   Φ 2 , and Φ 3 , it had a much lower tolerance for even a small proportion of extreme opposite opinions than Φ 1 ,   Φ e ,   Φ 2 , and Φ 3 did.