Validity of four measures in assessing school canteen menu compliance with state-based healthy canteen policy

Issue addressed In order to assess the impact of healthy school canteen policies on food availability for students, valid methods of measuring compliance are needed that can be applied at scale. The aim of this study is to assess the validity and direct cost of four methods to assess policy compliance: 1) principal and 2) canteen manager self-report via a computer-assisted telephone interview; and 3) comprehensive and 4) quick menu audits by dietitians, compared with observations. Methods A cross-sectional study took place in the Hunter region of NSW, Australia, in a sample of 38 primary schools that had previously participated in a randomised controlled trial to improve healthy canteen policy compliance. Policy compliance was assessed using the four methods specified above. Percentage agreement, kappa, sensitivity and specificity compared with observations was calculated together with the direct time taken and costs of each method. Indirect costs (including set-up costs) for all measures have not been included. Results Agreement with observations was substantial for the quick menu audit (kappa=0.68), and moderate for the comprehensive menu audit (kappa=0.42). Principal and canteen manager self-report resulted in poor agreement and low specificity with the gold standard. The self-reported measures had the lowest cost, followed by the quick menu audit and lastly the comprehensive menu audit. Conclusion The quick menu audit represents a valid and potentially low-cost method of supporting policy implementation at scale. So what? This study demonstrates that a quick menu audit represents a valid measure of undertaking assessment of school canteen policy compliance at a population level.