Inter-rater reliability of a composite health promotion scoring system developed in Singapore

In view of the important role of the environment in improving population health, implementation of health promotion programmes is recommended in living and working environments. Assessing the prevalence of such community health-promoting practices is important to identify gaps and make continuous and tangible improvements to health-promoting environments. We aimed to evaluate the inter-rater reliability of a composite scorecard used to assess the prevalence of community health-promoting practices in Singapore.

Inter-rater reliability for the use of the composite health promotion scorecards was evaluated in eight residential zones in the western region of Singapore. The assessment involved three raters, and each zone was evaluated by two raters. Health-promoting practices in residential zones were assessed based on 44 measurable elements under five domains - community support and resources, healthy behaviours, chronic conditions, mental health and common medical emergencies - in the composite scorecard using weighted kappa. The strength of agreement was determined based on Landis and Koch's classification method.

A high degree of agreement (almost perfect-to-perfect) was observed between both raters for the measurable elements from most domains and subdomains. An exception was observed for the community support and resources domain, where there was a lower degree of agreement between the raters for a few elements.

The composite scorecard demonstrated a high degree of reliability and yielded similar scores for the same residential zone, even when used by different raters.