|
This tutorial surveys new emerging connections between research in manifold learning and Markov decision processes and reinforcement learning. The tutorial will also introduce diffusion wavelets , a novel class of multiresolution wavelet-based manifold learning methods which are not well known in the machine learning community. Together, the Laplacian and wavelet based manifold learning methods hold the promise of a new generation of powerful tools for solving MDPs and RL, including ways of approximating value functions that respect geodesic distances on the underlying manifold; faster methods of policy evaluation and novel variants of policy iteration where both the representation and optimal policy can be simultaneously learned; algorithms for hierarchical reinforcement learning where the underlying hierarchy is automatically learned; novel approaches to transfer learning by transferring shared representations; and enabling reinforcement learning methods without requiring (task-specific) rewards.
The tutorial will be accesssible to researchers and graduate students working in any area of machine learning or related areas (robotics, statistics etc.). The tutorial will include a detailed introduction to the underlying mathematics as well as describe basis construction algorithms and how these lead to novel ways of solving MDPs and RL problems. All the ideas will be illustrated using hands-on MATLAB demonstrations.