An examination of graduate students' statistical judgments: statistical and fuzzy set approaches

The present study examined how statistical significance levels are treated and interpreted by graduate students who use hypothesis-testing in their scientific investigation. To test underlying psychological aspects of hypothesis-testing, the idea of fuzzy set theory was employed to identify the uncertain points in judgments. 34 graduate students in a psychology department made judgments about hypothetical statistical decisions. The results indicated that (1) the majority of these students treated significance levels on a continuum and rated them according to the magnitude of statistical significance; (2) the subjects shifted their decisions based on the types of hypothetical scenarios but not by the sample sizes; instead, they interpreted a smaller sample size as being less reliable. (3) The subjects frequently chose formally used statistical terms, e.g., Significant and Not Significant, more than graduated verbal expressions, e.g., Marginally Significant and Borderline Significant; and (4) the Fuzziness (degree of confidence in decision-making) was dependent on individuals and existed more in the critical points of transition where judgments are most difficult. The Fuzziness Index illustrated the subtle shifts of human decision-making patterns in statistical judgments. Underlying decision uncertainties and difficulties can be illustrated by functions generated from fuzzy set theory, which may more closely resemble human psychological mechanism. This integrative study of fuzzy set theory and behavioral measurements appears to provide a technique that is more natural for examining and understanding imprecise boundaries of human decisions.