Rule fitness and pathology in learning classifier systems

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Traditional strength-based classifier systems exhibit poor adaptation and generalization in certain reinforcement learning tasks.
• Accuracy-based XCS classifier systems demonstrate superior adaptation and generalization capabilities.

Purpose of the Study:

• To analyze the differences in adaptation and generalization between strength-based and accuracy-based classifier systems.
• To introduce and define 'strong overgeneral' and 'fit overgeneral' rules.
• To identify the weaknesses of each system and propose methods for improvement.

Main Methods:

• Development of a taxonomy for rule types within classifier systems.
• Qualitative analysis through extreme simplification of classifier systems.
• Investigation of rule behavior under different fitness-based mechanisms (strength vs. accuracy).
• Analysis of reward and value function biases.

Main Results:

• Strong overgeneral rules are identified as a critical weakness in strength-based systems.
• Strong overgeneral rules are detrimental in XCS but can be fit in strength-based systems (SB-XCS).
• Fit overgeneral rules can be designed for XCS by introducing biases in reward variance, highlighting system-specific vulnerabilities.

Conclusions:

• The performance differences between strength-based and accuracy-based systems are attributed to their distinct handling of overgeneral rules.
• Reward function biases significantly influence the behavior of strong overgeneral rules.
• XCS can be optimized by introducing reward variance biases, while SB-XCS has inherent weaknesses related to strong overgeneral rules.