Demystifying Fisher Information: What Observation Data Reveal about Our Models.

Area of Science:

• Information theory
• Computational modeling
• Geoscience

Background:

• Information theory provides a framework for understanding data transmission and uncertainty.
• Observation data is crucial for comparing parameter estimates and predictions across different models.
• Jacobian Information (JI) and Fisher Information (FI) are key metrics derived from model sensitivity and disorder.

Purpose of the Study:

• To investigate the relationship between Jacobian Information (JI) and Fisher Information (FI).
• To demonstrate how model complexity, structure, boundary conditions, and over-fitting impact parameter and prediction uncertainty.
• To quantify uncertainty in model estimates and predictions using information-theoretic measures.

Main Methods:

• Utilized one-dimensional models to explore information-theoretic concepts.
• Quantified Jacobian Information (JI) using the determinant of the weighted Jacobian (sensitivity) matrix.
• Quantified Fisher Information (FI) using the determinant of the weighted FI matrix, relating it to model entropy.

Main Results:

• Increased model complexity led to higher JI and greater parameter/prediction uncertainty.
• Fisher Information (FI) generally increased with complexity, but decreased with significant model error.
• Models with lower FI exhibited higher disorder (entropy) and increased uncertainty.
• Constant-head boundary conditions resulted in lower JI and FI compared to constant-outflow boundaries.
• Over-fitted models showed reduced JI and FI due to insufficient data for parameter estimation.

Conclusions:

• Model complexity directly influences information content and associated uncertainties.
• Fisher Information (FI) serves as a valuable indicator of model disorder and reliability.
• Boundary conditions and model overfitting significantly affect the information captured by JI and FI, impacting predictive accuracy.