Identifying Useful Subgoals in Reinforcement Learning by Local Graph Partitioning
Özgür Şimşek, Alicia P. Wolfe, Andrew G. Barto
In Proceedings of the Twenty-Second International Conference on Machine Learning (ICML-05), forthcoming.
We present a new subgoal-based method for automatically creating useful skills in reinforcement learning. Our method identifies subgoals by partitioning local state transition graphs—those that are constructed using only the most recent experiences of the agent. The local scope of our subgoal discovery method allows it to successfully identify the type of subgoals we seek—states that lie between two densely-connected regions of the state space—while producing an algorithm with low computational cost.