COMP 875: Machine Learning Techniques in Image Analysis

Fall 2009, Tuesdays and Thursdays 3:30-4:45, FB 009

Instructor: Svetlana Lazebnik  (lazebnik -at-

Quick links: schedule, reading materials, useful resources

About the Course

This is a graduate seminar course on statistical and machine learning techniques. It is intended as a standard part of the medical imaging and computer vision graduate curriculum, to be taken by students who have already taken COMP 665, 775, or 776. However, students from other research areas who want to get a better background in machine learning are also encouraged to attend. For example, students who have taken a Data Mining course would be well prepared for this one.

The course will consist of a series of introductory lectures, followed by student presentations. The selection of presentation topics will be driven by the interests of students who are enrolled. Possible topics include, but are not limited to, the following:

  • Classifiers: linear models, support vector machines, boosting
  • Kernel methods
  • Bayesian methods, EM
  • Random field models
  • Sampling techniques such as Markov Chain Monte Carlo
  • Unsupervised learning: density estimation, clustering
  • Manifold learning and dimensionality reduction
  • Distance metric learning
  • Semi-supervised learning
  • Online and active learning
  • Sequential inference (i.e., tracking)
  • Large-scale learning
For more detail, see this list of reading materials.


Each student will be required to make one or two presentations (depending on enrollment) on his or her chosen topic (see this list for possible topic ideas). The presentations should be created with the following criteria in mind:
  • Integration: think of yourself as professor for a day. You should strive to give a comprehensive and understandable lecture introducing the class to a specific topic in machine learning. Even if your presentation centers on one particular method or application, you need to consult multiple sources and give a survey of relevant literature.

  • Critical thinking: you are expected to give a balanced and critical perspective on the topic you are presenting. You should go beyond simply giving a list of relevant papers or describing the techniques. Compare and contrast different papers, question assumptions, expose possible flaws in experimental protocol, identify limitations, suggest alternative applications and/or directions for future research.

  • Interactivity: try to involve the class. Ask for input from the class, try to think of discussion topics. This part should follow naturally if you get in the "critical thinking" mindset.
Here is an example of a good presentation from a similar course at UT Austin. To help everybody prepare a high-quality presentation, there will be a couple of intermediate steps. In the first few weeks of the course, you will be required to submit a reading list for your presentation (this may be combined with your project proposal, see below). A week before you are scheduled to present, you will be required to send me a preliminary version of your slides and meet with me to discuss it.


It is highly recommended that you build your project around the same topic as your presentation (this will save you a lot of effort in the course). You have two options:
  • Implementation: do an empirical evaluation of one or more methods, apply an existing method to an interesting problem, extend an existing method, or design and implement your own solution to a learning problem.

  • Survey paper: you can think of this as an extended set of notes produced based on your presentation slides. It should serve as a thorough introduction for anyone who wishes to learn about your topic. The survey should be structured as a formal academic paper. It should be 10-15 pages in length (single-spaced, single colum, 11pt) and typeset in LaTeX.
Deliverables throughout the semester will consist of a proposal (which may be integrated with the reading list for your presentation), one progress report (in the case of a survey paper, this will be a first draft), and a final report.


  • Participation and attendance: 30%
  • Presentation (including reading list, preliminary slides and meeting, and final slides): 35%
  • Project (including proposal, progress report, and final report): 35%


Date Topic Notes and readings
August 25 First class meeting: PPT, PDF  
August 27 Intro to machine learning: PPT, PDF  
September 1 Statistical decision theory: PDF
Guest lecturer: Maxim Raginsky
Reading: Hastie, Tibshirani & Friedman (HTF) ch. 2
September 3 Linear methods for classification: PDF
Guest lecturer: Maxim Raginsky
Reading: HTF sec. 4.4, 4.5
Presentation reading lists due
September 8 Support vector machines: PDF
Reading: Burges tutorial
September 10 SVMs concluded: PDF  
September 15 Nonparametric methods: PDF  
September 17 Mixture modeling, EM: PDF Reading: Bishop ch. 9
September 22 Learning techniques for images and text: PPT
Presenter: Rahul Raguram
Rahul's reading list
September 24 Transformation-based learning: PDF
Presenter: Brendan Walters
Brendan's reading list
September 29 Sequence classification: PDF
Presenter: Andrew White
Andrew's reading list
Project proposals due (instructions)
October 1 Inferring Information from Encrypted Network Data: PDF
Presenter: Kevin Snow
Kevin's reading list
October 6 Breaking CAPTCHAS: PDF
Presenter: Xiaoyang Wen
Xiaoyang's reading list
October 8 Manifold Kernel Regression: PDF
Presenter: Gabe Hart
Gabe's reading list
October 13 Clustering techniques for image segmentation: PPT
Presenter: Liang Shan
Liang's reading list
October 15 Estimating redshift of quasars: PDF
Presenter: Danny Kumar
Danny's reading list
October 20 Registration of deformable anatomic structure: PDF
Presenter: Chen-Rui Chou
Chen-Rui's reading list
October 27 Learning of distance functions: PDF
Presenter: Tim Johnson
Tim's reading list
October 29 Locality sensitive hashing and large-scale image search: PPT
Presenter: Yunchao Gong
Yunchao's reading list
November 3 Min-hash for near-duplicate image search: PDF
Presenter: Jonathan Bidwell
Jonathan's reading list
Project progress reports due (instructions)
November 5 Random fields for image labeling: PPT
Presenter: Yilin Wang
Yilin's reading list
November 10 Comparing distributions: PDF
Presenter: Matthew O'Meara
Matt's reading list
November 12 Gibbs sampling for motif finding and sequence alignment: PPT
Presenter: Christopher Sheldahl
Christopher's reading list
November 17 Action recognition: PPT
Presenter: Sami Benzaid
Sami's reading list
November 19 Malware classification and clustering: PPT
Presenter: Peng Li
Peng's reading list
November 24 Semi-supervised learning: PDF
Presenter: Svetlana Lazebnik
Reading: J. Zhu survey
December 1 Stanford Autonomous Helicopter: PPT
Presenter: Joe Tighe
Joe's reading list
December 3 Review and course wrap-up Final project reports and surveys due by 5PM (instructions)
December 8 No class (NIPS)  

Some useful links