Assessing semantic coherence in conditional probability estimates

Semantic coherence is a higher-order coherence benchmark that assesses whether a constellation of estimates--P(A), P(B), P(B | A), and P(A | B)--maps onto the relationship between sets implied by the description of a given problem. We present an automated method for evaluating semantic coherence in conditional probability estimates that efficiently reduces a large problem space into five meaningful patterns: identical sets, subsets, mutually exclusive sets, overlapping sets, and independent sets. It also identifies three theoretically interesting nonfallacious errors. We discuss unique issues in evaluating semantic coherence in conditional probabilities that are not present in joint probability judgments, such as errors resulting from dividing by zero and the use of a tolerance parameter to manage rounding errors. A spreadsheet implementing the methods described above can be downloaded as a supplement from www.springerlink.com .