Recommendations on statistics and benchmarks to assess photochemical model performance

Area of Science:

• Air Quality Modeling
• Environmental Science
• Atmospheric Chemistry

Background:

• Photochemical grid models are vital for air quality assessments but lack consistent performance evaluation metrics.
• Inconsistent evaluations hinder quantitative comparisons between models and understanding of uncertainties.

Purpose of the Study:

• To develop updated, quantitative performance benchmarks for ozone and particulate matter (PM) concentrations.
• To promote consistent evaluation procedures and metrics across diverse modeling applications and scales.
• To guide improvements in photochemical modeling for better air quality policy assessments.

Main Methods:

• Analyzed a decade of North American photochemical modeling studies.
• Developed new benchmarks (goals and criteria) for three statistical metrics.
• Considered spatial scales from urban to regional and temporal scales from episodic to seasonal.

Main Results:

• Established updated quantitative performance benchmarks for ozone and PM.
• Recommended specific statistical metrics, evaluation procedures, and graphical methods.
• Provided a framework for assessing future changes in model performance.

Conclusions:

• Consistent and updated benchmarks are essential for reliable photochemical model evaluation.
• Improved model evaluations enhance understanding of air quality, model uncertainties, and inform regulatory decisions.
• Periodic revision of benchmarks is necessary to adapt to evolving model capabilities and air quality characteristics.