Autonomous agents, whether biological or synthetic, are faced with a difficult sequential decision-making task: how to act reliably to achieve long-term goals despite significant uncertainty in perception and action?

Using recent developments in the field of machine learning and adaptive control, this talk will describe a general approach that uses hierarchical spatio-temporal abstraction to simplify the problem of deciding how to act. Perception, action, and memory are modeled in a unified way as temporally extended processes, amenable to adaptation and control by agents through a process of interactive learning with their environment. A general statistical framework based on hierarchical Markov models provides a theoretical underpinning for modeling temporally extended processes. This framework also facilitates exploiting task and behavioral hierarchy and modularity. Control is achieved through greedy action selection, based on maximizing values that predict long-term outcomes under uncertainty.

Drawing upon recent work of my research group, we will present several case studies that illustrate the interdisciplinary scope of this approach, ranging from models of visual attention and recognition in humans and gaze control in machines, to multi-agent coordination in manufacturing and scheduling, and spatial navigation in insects and robots.