Rasch Analysis of the Composite Autonomic Symptom Score (COMPASS-31) From Data Collected in an Online Population

A Dutch internet panel with volunteers was assessed using the patient-reported outcome measure known as the Composite Autonomic Symptom Score (COMPASS-31).

This study aimed to analyze the Dutch version of the COMPASS-31 using classical test theory and Rasch analysis, or one-parameter item response theory, to examine the questionnaire's structure for the second time in a different population. The first group consisted of clinical patients, while the second group consisted of the general population. The choice of the general population contrasts with most validation studies, which focus on clinical groups, as in our first study.

A transversal observational study of a cohort of 100 people voluntarily completed the Dutch version of the COMPASS-31 online. The sample population comprised 100 persons (42 females, 58 males). The age groups and their percentages were as follows: 18-29 years, 37%; 30-44 years, 22%; 45-60 years, 31%; and over 60 years, 10%. Classical test theory was performed after importing the data into IBM SPSS Statistics for Windows, Version 31.0 (IBM Corp., Armonk, New York, United States). Six Rasch analyses were completed for each domain of the COMPASS-31 using the Winsteps measurement software (Winsteps Rasch Measurement Version 5.10.0).

The COMPASS-31 domains performed moderately in a group of adult volunteers. Cronbach's alpha across the domains was below 0.65, except for the Gastrointestinal, Bladder, and Pupillomotor domains, which were 0.65 or above. The person's reliability is below 0.8, the minimum level required for serious decision-making. Item reliability was good, except for the Bladder domain, which had a reliability of zero. Only the Pupillomotor domain can differentiate between the two population groups. Not all items in the Vasomotor and Gastrointestinal domains measure a single unidimensional variable, one of the prerequisites for a Rasch model. The probability curve for different domains performed adequately. Patients were able to distinguish between four levels of items. Some domains included categories that were too broad.

This study, conducted with a different group of internet volunteers, offered several valuable insights into the psychometric properties of the COMPASS-31 and its domains. Our data again did not align with the Rasch model. The analysis failed to confirm the scale's reliability, as the domains could differentiate only one or two levels of autonomic symptoms in adults. The person's reliability is below 0.8, which is the minimum threshold needed for meaningful decision-making. Only the Pupillomotor domain can distinguish between the two groups, while the other domains identify only one of them. The COMPASS-31 is a screening tool or descriptive profile, not a unidimensional measure designed to produce a single, interval-level score for any domain. Based on our findings, we recommend against using the total score as a single metric. The domains should serve as independent, ordinal indices. Item banks for computer adaptive testing (CAT) should be developed specifically for each domain.