Decision making for large-scale multi-armed bandit problems using bias control of chaotic temporal waveforms in...

Area of Science:

• Photonics
• Reinforcement Learning
• Complex Systems

Background:

• The multi-armed bandit problem is crucial for reinforcement learning but faces scalability challenges with current photonic technologies.
• Existing photonic decision-making methods are not yet optimized for large-scale applications.

Purpose of the Study:

• To numerically investigate a photonic decision-making strategy for large-scale multi-armed bandit problems.
• To explore the use of chaotic temporal waveforms from semiconductor lasers for enhanced decision-making.

Main Methods:

• Generating chaotic temporal waveforms from semiconductor lasers with optical feedback.
• Assigning each waveform to a slot machine in the multi-armed bandit problem.
• Adjusting waveform amplitudes using the tug-of-war method based on rewards and selecting the maximum-amplitude waveform.

Main Results:

• Examined scaling properties by increasing slot machines to 1024, achieving a scaling exponent of 0.97.
• Demonstrated that the proposed photonic method outperforms existing software algorithms in scaling exponent.
• Validated the effectiveness of controlling chaotic waveform biases for decision-making.

Conclusions:

• The developed photonic decision-making method effectively addresses large-scale multi-armed bandit problems.
• This research opens avenues for photonic accelerators in complex decision-making tasks.
• The findings suggest a promising direction for future photonic implementations in artificial intelligence.