On different facets of regularization theory
Related Experiment Video
Updated: Sep 28, 2026

Generating Strictly Controlled Stimuli for Figure Recognition Experiments
Published on: March 18, 2019
1Adaptive Systems Lab, Communications Research Laboratory, McMaster University, Hamilton, Ontario, Canada L8S 4K1. zhechen@soma.crl.mcmaster.ca
Abstract:
This review provides a comprehensive understanding of regularization theory from different perspectives, emphasizing smoothness and simplicity principles. Using the tools of operator theory and Fourier analysis, it is shown that the solution of the classical Tikhonov regularization problem can be derived from the regularized functional defined by a linear differential (integral) operator in the spatial (Fourier) domain. State-of-the-art research relevant to the regularization theory is reviewed, covering Occam's razor, minimum length description, Bayesian theory, pruning algorithms, informational (entropy) theory, statistical learning theory, and equivalent regularization. The universal principle of regularization in terms of Kolmogorov complexity is discussed. Finally, some prospective studies on regularization theory and beyond are suggested.
Related Concept Videos
Application of Linearization and Approximation
Linearization and Approximation
State Function, Exact and Inexact Differentials
Partial Differential Equations
Law of Rational Indices
Application of Nonlinear Inequalities