A framework for evaluating OSH program effectiveness using leading and trailing metrics

Many employers and regulators today rely primarily on a few past injury/ illness metrics as criteria for rating the effectiveness of occupational safety and health (OSH) programs. Although such trailing data are necessary to assess program success, they may not be sufficient for developing proactive safety, ergonomic, and medical management plans.

The goals of this pilot study were to create leading metrics (company self-assessment ratings) and trailing metrics (past loss data) that could be used to evaluate the effectiveness of OSH program elements that range from primary to tertiary prevention. The main hypothesis was that the new metrics would be explanatory variables for three standard future workers compensation (WC) outcomes in 2003 (rates of total cases, lost time cases, and costs) and that the framework for evaluating OSH programs could be justifiably expanded. For leading metrics, surveys were developed to allow respondents to assess OSH exposures and program prevention elements (management leadership/ commitment, employee participation, hazard identification, hazard control, medical management, training, and program evaluation). After pre-testing, surveys were sent to companies covered by the same WC insurer in early 2003. A total of 33 completed surveys were used for final analysis. A series of trailing metrics were developed from 1999-2001 WC data for the surveyed companies. Data were analyzed using a method where each main 2003 WC outcome was dichotomized into high and low loss groups based on the median value of the variable. The mean and standard deviations of survey questions and 1999-2001 WC variables were compared between the dichotomized groups. Hypothesis testing was performed using F-test with a significance level 0.10.

Companies that exhibited higher musculoskeletal disorder (MSD) WC case rates from 1999-2001 had higher total WC case rates in 2003. Higher levels of several self-reported OSH program elements (tracking progress in controlling workplace safety hazards, identifying ergonomic hazards, using health promotion programs) were associated with lower rates of WC lost time cases in 2003. Higher reported exposures to noise and projectiles were also associated with higher rates of WC cases and costs in 2003.

This research adds to a growing body of preliminary evidence that valid leading and trailing metrics can be developed to evaluate OSH effectiveness. Both the rating of OSH efforts and the regular trending of past loss outcomes are likely useful in developing data-driven improvement plans that are reactive to past exposures and proactive in identifying system deficiencies that drive future losses.