Learning feature spaces for regression with genetic programming.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Intelligence

Background:

• Genetic programming (GP) is effective for feature learning in regression and classification.
• Multidimensional genetic programming (MGP) represents solutions as sets of programs, enabling better building block identification.
• Existing GP architectures vary in their ability to leverage evolutionary information for heuristic search.

Purpose of the Study:

• To investigate methods for guiding evolutionary search towards useful and complementary feature spaces in MGP.
• To explore the impact of new objectives and specialized semantic variation operators on GP performance.
• To assess the collinearity and complexity of data representations generated by different methods.

Main Methods:

• Discussed MGP architecture and its information utilization during evolutionary processes.
• Investigated biasing program components to guide search towards desired feature spaces.
• Introduced new objectives and specialized semantic variation operators, including a semantic crossover operator based on stagewise regression.

Main Results:

• The semantic crossover operator based on stagewise regression significantly improved performance on regression problems.
• The inclusion of semantic crossover yielded state-of-the-art results in a benchmark study of open-source regression problems.
• Compared favorably against several state-of-the-art machine learning approaches and other GP frameworks.

Conclusions:

• Specialized semantic variation operators, particularly semantic crossover, are effective for enhancing GP feature learning.
• The proposed methods lead to significant improvements in regression tasks and achieve competitive results.
• Further analysis is needed to assess the relevance, conciseness, and independence of the produced data representations.