Energy Efficient Execution of POMDP Policies

Area of Science:

• Artificial Intelligence
• Robotics
• Computer Science

Background:

• Advanced planning techniques for partially observable Markov decision processes (POMDPs) typically require substantial computation between decisions.
• The rise of mobile and embedded devices necessitates planning methods that operate under severe computational constraints.

Purpose of the Study:

• To develop techniques for compiling POMDP policies into efficient, executable controllers for resource-constrained devices.
• To explore controller compression methods for further optimization and improved performance.

Main Methods:

• Policy compilation into table-lookup controllers using alpha vectors or simulation-based approaches.
• Controller compression by removing redundant and dominated nodes, including consideration of stochastic controllers.
• Empirical evaluation on benchmark problems and a mobile Alzheimer's way-finding application.

Main Results:

• Two distinct techniques for compiling POMDP policies into approximately equivalent controllers were developed.
• Controller compression techniques were shown to yield smaller, more effective controllers.
• Finite-state controllers demonstrated the lowest battery consumption compared to other POMDP policies on mobile devices.

Conclusions:

• Compiled finite-state controllers offer a viable solution for deploying advanced AI planning on battery-limited mobile and embedded systems.
• The developed compilation and compression techniques enable efficient POMDP policy execution in real-world applications with strict computational and energy budgets.