Performance of the MARS-crop yield forecasting system for the European Union: Assessing accuracy, in-season, and...

19,980 crop yield forecasts have been published for the European Union (EU) Member States (MS) during 1993-2015 using the MARS-Crop Yield Forecasting System (MCYFS). We assess the performance of these forecasts for soft wheat, durum wheat, grain maize, rapeseed, sunflower, potato and sugar beet, and sought to answer three questions. First, how good has the system performed? This was investigated by calculating several accuracy indicators (e.g. the mean absolute percentage error, MAPE) for the first forecasts during a season, forecasts one month pre-harvest, and the end-of-campaign (EOC) forecasts during 2006-2015 using reported yields. Second, do forecasts improve during the season? This was evaluated by comparing the accuracy of the first, the pre-harvest, and the EOC forecasts. Third, have forecasts systematically improved year-to-year? This was quantified by testing whether linear models fitted to the median of the national level absolute relative forecast errors for each crop at EU-12 (EU-27) level from 1993 to 2015 (2006-2015) were characterized by significant negative slopes. Encouragingly, the lowest median MAPE across all crops is obtained for Europe's largest producer, France, equalling 3.73%. Similarly, the highest median MAPE is obtained for Portugal, at 14.37%. Forecasts generally underestimated reported yields, with a systematic underestimation across all MS for soft wheat, rapeseed and sugar beet forecasts. Forecasts generally improve during the growing season; both the forecast error and its variability tend to progressively decrease. This is the case for the cereals, and to a lesser extent for the tuber crops, while seasonal forecast improvements are lower for the oilseed crops. The median EU-12 yield forecasts for rapeseed, potato and sugar beet have significantly (p-value < 0.05) improved from 1993 to 2015. No evidence was found for improvements for the other crops, neither was there any significant improvement in any of the crop forecasts from 2006 to 2015, evaluated at EU-27 level. In the early years of the MCYFS, most of the yield time series were characterized by strong trends; nowadays yield growth has slowed or even plateaued in several MS. In addition, an increased volatility in yield statistics is observed, and while crop yield forecasts tend to improve in a given year, in recent years, there is no evidence of structural improvements that carry-over from year-to-year. This underlines that renewed efforts to improve operational crop yield forecasting are needed, especially in the light of the increasingly variable and occasionally unprecedented climatic conditions impacting the EU's crop production systems.