Frequency Principle in Broad Learning System.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Signal Processing

Background:

• Deep neural networks (DNNs) excel but require extensive training time due to complex structures and numerous parameters.
• A fast and efficient alternative, the discriminative broad learning system (BLS), leverages a flat structure and incremental learning for classification and regression.
• Previous research highlighted BLS's performance but lacked explanation for its generalization capabilities.

Purpose of the Study:

• To interpret the generalization ability of the broad learning system (BLS) from a frequency domain perspective.
• To uncover the underlying principles governing BLS's efficiency and effectiveness.
• To provide insights into the frequency-based learning mechanism of BLS.

Main Methods:

• Analysis of the broad learning system (BLS) through the lens of frequency domain interpretation.
• Investigating the learning process concerning the addition of feature nodes and enhancement nodes.
• Examining how BLS handles different frequency components during incremental learning.

Main Results:

• Discovery of the 'frequency principle' in BLS: it preferentially captures low-frequency components rapidly.
• BLS subsequently fits high-frequency components during its incremental learning process.
• This frequency-based learning contributes to BLS's strong generalization performance.

Conclusions:

• The identified frequency principle offers a novel explanation for BLS's effective generalization.
• Understanding this principle can guide the expansion of BLS applications.
• The findings suggest BLS's potential for efficient learning across various domains by exploiting frequency characteristics.