CS370: Introduction to Computer Vision

• Spring 2012; Monday and Wednesday, 10:35-11:50 • Computer Science Building, Room 140

Instructor

Erik Learned-Miller
elm at cs.umass.edu
(413) 545-2993

Teaching Assistant

Laura Sevilla Lara
lsevilla at cs.umass.edu
TA Hours: Tuesdays 3PM - 5PM in Room 256.
Please, write "CS370" as the subject of any email regarding the class or containing assignments.

Prerequisites

Assignments

Need to be emailed to Laura only.

Reading Materials

Textbook: Computer Vision: Algorithms and Applications, by Richard Szeliski: On-line copy available here

I do NOT recommend buying the textbook unless you want it for your own purposes. We will use it some in this course, but not a lot. You should be able to get by the on-line version.

Resources

MATLAB Tutorial

Interesting Links

Beau Lotto Ted Talk on optical illusions

Checker shadow illusion

Early color photographs by S. M. Prokudin-Gorsky

Flower Garden movie.

Non-lambertian reflectance functions

Explanation of hexagonal sampling efficiency

Color Blind Test

Problem Sets

Homework 3 Sample Solution

Description

Schedule
Date Lecture topic New assignments Assignments due Reading
Sept. 5 UNIT 1: Introduction. What is Computer Vision? What are the goals of computer vision? What can we learn by studying the human vision system?

Introduction to MATLAB.
Assignment 1: Read Lightness Perception and Lightness Illusions
Come up with five questions relevant to the paper. These can be things you didn't understand after a careful reading of the paper, or questions which the paper raises. Turn in the answer written up as a .pdf file. You will be graded on the depth of your questions and how much thought you were judged to have put into them.

As. 1 due Sep. 12

Introduction to computer vision
Sept. 10 Introduction to using MATLAB for Computer Vision. Matlab Session Transcript from Lecture


Assignment 2: Colorizing the Prokudin-Gorsky photo collection

As. 2 due Sept. 17

Chap. 2 in Textbook (skim)
Sept. 12 Introduction to color. The electromagnetic spectrum. The multi-frequency nature of light. The relative absorption of light by surfaces. The basic anatomy of the eye. Rods and cone cells in the retina. The relative sensitivity of rods and short, medium, and long wavelengh cone cells. Computing the response of cone cells to a particular distribution of light. Lecture slides

David Heeger's slides on trichromacy




Beau Lotto Ted Talk on optical illusions
Sept. 17 Finish discussion of metamers. See this document.

UNIT 2: Probability, Statistics, and Learning Formalizing the decision making process. Minimizing error. Maximizing utility. Review of basic probability theory. You will be responsible for all of the basic probability theory in this handout.




Probability handout (see lecture description).
Sept. 19 Basic probability continued. Joint probability. Marginalization. Conditional Probablity. Bayes rule.




Sept. 24 Introduction to supervised learning. Supervised learning for vision. Estimating distributions from data. Features of images.



Handout: Supervised learning and estimation
Sept. 26 Estimating distributions from data. Estimating joint distributions of multiple variables. Leveraging independence for better estimation. More applications of Bayes' rule.

Assignment 3. Classification of handwritten digits. Training and test data file (digits.mat)



Oct. 1 Independence of features. Implications for estimating distributions. Consequences of assuming independence when variables are not independent. Comparing the dependence of features using mutual information. Evaluating the value of features using mutual information.




Oct. 3 UNIT 3: Alignment
Basic transformation classes: translation, rotation, rigid, similarity, affine, perspective, diffeomorphisms, arbitrary.

Applying transformations to images: Forward versus inverse transformations.

Optimization of alignment criteria: exhaustive, keypoint, and local. Gradient descent for local alignment.
Alignment by maximization of mutual information.


Assignment 4. Aligning the Prokudin-Gorsky images automatically.
Due October 17

Mutual Information handout.
Oct. 9 Alignment continued.




Oct. 10 More alignment.
Alignment by correlation demo in Matlab.


Intro to alignment slides.




Oct. 15 Solving difficult alignment problems through joint alignment: congealing. Introduction to the "self likelihood", and its relation to entropy. The self-likelihood as a criterion of joint alignment.
Joint alignment slides.




Oct. 17 Congealing continued.

The congealing page.




Oct. 22 Finish congealing. Finish Alignment Unit.

UNIT 4: Features What are features? Principles of feature engineering. Optimal features. Commonly used features. Features vs. representations.




Oct. 24 Features continued.


Assignment 5. Problem Set due Wednesday, October 31st, by class time.

Oct. 29 NO CLASS: Hurricane SANDY cancelled class.




Oct. 31 Features: Linear filters and convolution.
Exam review




Nov. 5 **************************************** In Class Mid-term: Covers material up to and including UNIT 3 (Oct. 22nd and earlier) ***************************************




Nov. 7 Linear filters: Correlation filters for matching and tracking.




Nov. 12 VETERAN'S DAY: NO CLASS




Nov. 14 Keypoint matching: Scale Invariant Feature Transforms (SIFT)




Nov. 19 Michal Erel's SIFT slides used in class
Matlab scale space code from class
SIFT features, continued. Matlab topics: Gaussians, Gaussian convolution, building a scale space, building a Difference-of-Gaussian (DOG) scale space. Also, finding local maxima and minima in DOG scale space. Calculating gradients in an image, magnitudes of gradients, and angles of gradients.


Assignment: Correlation tracking
Due, December 3rd (revised from November 28th).
MATLAB solution to correlation tracking
Nov. 21 Using SIFT for panorama stitching. RANdom Sample Consensus (RANSAC). Slides from lecture can be found here. See lecture number 13 on RANSAC.



THURSDAY
Nov. 22
THANKSGIVING



Nov. 26 UNIT 5: Faces
Intro to face recognition. Intro to face slides


Nov. 28 Viola-Jones face detection. Haar features. Boosting. Integral images.


Dec. 3 Viola-Jones continued. The Boosting Cascade. Viola-Jones face detection. Slides by Lana Lazebnik, adapted from Paul Viola.


Dec. 5 Principal Components Analysis and Morphable models. Morphable models for acromegaly detection.
Acromegaly slides


MONDAY, DECEMBER 10, 1:30-3:30, Room 142 Computer Science building FINAL EXAM

Review for FINAL