CMPSCI 697: Deep Learning: Fall 2015:

Professor Sridhar Mahadevan

Friday 9:05-12:00


Room 142, CS Building

 
 

Deep learning is a recent breakthrough in the field of machine learning that has become highly popular, due in large part to its success at solving extremely difficult high-dimensional problems, ranging from computer vision and speech recognition, to natural language processing and reinforcement learning. Large groups have formed at companies ranging from Baidu, Facebook, Google, IBM, Microsoft, and scores of smaller startups. This course will provide a state-of-the-art introduction to both the theory and practice of deep learning. The course can be categorized broadly into the following types: (i) Historical background, previous work in deep learning neural networks (ii) The problem of representation discovery, and linear/nonlinear approaches (iii) Deep learning models: autoencoders, restricted Boltzmann machines, deep belief networks, convolutional networks, and feedforward networks. (iv) Algorithms for training deep networks (iv) Applications of deep learning to vision, speech, natural language, and reinforcement learning. (v) Software packages for training deep networks (vi) GPU-based methods for training deep network (vii) Theory of deep networks, including algorithmic stability and theoretical capabilities

  1. Historical review of perceptrons (from the 1960s) and feeforward neural networks (from the mid-1980s). Limitations of these models.

  2. The problem of representation discovery, and growing insights from the study of the brain and other areas in cognitive science.

  3. Deep learning architectures: stacked denoising autoencoders, deep belief networks, restricted Boltzmann machines, and convolutional networks.

  4. Gradient-based algorithms for training deep learning networks.

  5. Applications of deep learning to vision, speech, natural language, and reinforcement learning.

Overview of the course

Instructor: Professor Sridhar Mahadevan

Email: last name without the “n” AT cs DOT umass DOT edu

Room 203, CS Building

Office hours: By arrangement