Researcher judgment and study design: challenges of using administrative data.

Area of Science:

• Occupational health and safety
• Biostatistics
• Labor economics

Background:

• Concerns exist regarding the undercounting of workplace injuries and illnesses by the Bureau of Labor Statistics (BLS) and workers' compensation agencies.
• A previous study suggested BLS surveys capture only a portion of workplace injuries.
• A recent study indicated higher BLS reporting rates for workers' compensation cases in Minnesota.

Purpose of the Study:

• To examine the sensitivity of findings in two studies on workplace injury undercounting.
• To assess the impact of alternate sample definitions and study assumptions on reporting rate estimates.

Main Methods:

• The study re-evaluated two existing research datasets using alternative sample construction rules and analytical assumptions.
• Sensitivity analysis was performed to test the robustness of the findings under different methodological choices.

Main Results:

• Adjusting sample construction rules in an earlier study increased estimated BLS reporting rates from 68% to 77%.
• Modifying assumptions in the Minnesota study decreased the estimated BLS reporting rate from 90% to a range of 53-64%.

Conclusions:

• Linking administrative data from multiple sources involves significant judgment in data construction and analysis.
• Varying sample construction rules and assumptions yield different reporting rate estimates.
• Sensitivity analysis is crucial when methodological alternatives exist, and in this case, it supports the hypothesis of substantial underreporting of workplace injuries.