Q-learning with temporal memory to navigate turbulence.

Area of Science:

• Robotics and Artificial Intelligence
• Animal Behavior and Sensory Ecology

Background:

• Agents navigating turbulent environments often rely on olfactory cues.
• Lack of spatial perception necessitates learning strategies from odor stimuli alone.

Purpose of the Study:

• To investigate if agents can learn robust olfactory navigation in turbulent environments using sequential decision-making.
• To develop and test a reinforcement learning algorithm for odor-guided navigation.

Main Methods:

• Developed a reinforcement learning algorithm with interpretable olfactory states and temporal memory.
• Trained agents using realistic turbulent odor cues.
• Analyzed agent performance based on odor plume characteristics and learned strategies.

Main Results:

• Agents learned to navigate effectively using discretized olfactory states and temporal memory.
• An optimal memory strategy was identified, balancing plume detection and recovery.
• Learned recovery strategies involved cross-wind casting, mimicking insect behavior.

Conclusions:

• Two key olfactory trace features are sufficient for learning navigation in realistic odor plumes.
• Learned strategies are robust to environmental changes, suggesting adaptability.
• Reinforcement learning provides a viable framework for developing autonomous olfactory search agents.