Rethinking Model Transferability: Validity Domains as a New Approach to Delineate the Limits of Bloom Date...

Area of Science:

• Ecology
• Environmental Science
• Computational Biology

Background:

• Accurate prediction under novel environmental conditions is a key modeling challenge, especially without validation data.
• Model transferability is often limited by forecast horizons, necessitating new frameworks.
• Understanding model reliability across environmental gradients is crucial for climate change adaptation.

Purpose of the Study:

• To introduce and apply the concept of "validity domains" to assess model transferability beyond traditional forecast horizons.
• To compare the transferability of process-based and machine learning models using phenological data.
• To provide a framework for evaluating model applicability under shifting climate conditions.

Main Methods:

• Calibrated process-based and machine learning models using Japanese cherry blossom phenology data from 48 locations across a climate gradient.
• Validated models across all locations, interpolating performance metrics to create a predictive accuracy surface.
• Delineated model-specific validity domains based on calibration and application conditions along the environmental gradient.

Main Results:

• Process-based models showed broader validity when calibrated in colder environments but degraded in warmer conditions.
• Machine learning models exhibited narrower but more consistent validity across the temperature gradient.
• Model type and calibration environment significantly influenced predictive reliability under novel conditions.

Conclusions:

• Validity domains provide a robust framework for assessing model applicability under changing climates.
• Mapping these domains offers practical guidance for selecting appropriate models and understanding their limitations.
• The study quantifies how far models can be reliably used before predictions become inaccurate.