Poor Agreement Between Two Commonly Used Measures of Shame- and Guilt-Proneness

This study sought to assess the agreement between commonly used measures of shame- and guilt-proneness, the Test of Self-Conscious Affect-3 (TOSCA-3), representing scenario measures, and the Personal Feelings Questionnaire-2 (PFQ-2), representing checklist measures. To overcome the limitations of the widely used correlation analysis, the 2 measures were compared by the Bland-Altman method. We administered both measures at once to the same sample of 138 graduate students (67.39% were female; median age = 27 years). A randomly selected sample of 46 students repeated the procedure 8 weeks later. We tested how well our data fit the hypothesized measurement models, analyzed internal consistency of measures, evaluated their repeatability, and analyzed the agreement between them. To account for the different ranges, both measures' scores were expressed as the percentages of their maxima. The observed data fit the proposed models well. Both measures showed good internal consistency and repeatability. In the shame domain, TOSCA-3 exceeded PFQ-2 scores by 22.32% on average (49.81, -5.13%; 95% limits of agreement), and even more in the guilt domain, by 38.4% (67.75, 8.20%). Our results question the often-assumed congruence of the shame domains assessed by scenario and checklist measures.