COMP 590/776: Computer Vision in 3D World

Fall 2023

Instructor: Roni Sengupta
Tuesday and Thursday, 11:00-12:15pm, Sitterson Hall (SN) - Room 0011

[ Overview | Course Policies | Schedule ]

Image courtesy Computer Vision: Algorithms and Applications, 2nd ed., by Richard Szelski

Overview


Course Description

This is an advanced undergraduate and graduate level course focusing on the fundamentals of Computer Vision. Recently we have noticed an explosion in Computer Vision research and real world applications, creating a huge impact in society. This revolution has been mainly powered by the advances in Deep Learning techniques and computing resources. In this course we will focus on learning the fundamentals of Computer Vision, and we will try to answer the following questions: (a) How do we perceive color and how are images formed? (b) How do we process images and extract features from them? (c) How do we connect the 3D world to 2D images and reconstruct 3D from images? (d) How do we develop machine perception that recognize, detect, and segment objects? We will also study how some Computer Vision algorithms can introduce bias and cause harm to a certain portion of the population, often underrepresented communities, and how as researchers and practitioners we can do better.

Prerequisite

MATH 233 (or any multivariate calculus course) + MATH 347 (or any linear algebra course) + COMP 211 (or COMP 311) + COMP 301 (or COMP 411) + some basic knowledge of probability + coding in Python. There will be a linear algebra, calculus and probability review in one lecture followed by a short assignment. Your ability to handle this assignment will indicate your mathematical preparedness for the rest of the course. Knowledge of deep learning is helpful, but not required.

Resources

Computer Vision: Algorithms and Applications, 2nd ed., by Richard Szelski
Multiple View Geometry in Computer Vision , by Richard Hartley and Andrew Zisserman.
Computer Vision: A Modern Approach, by David Forsyth and Jean Ponce

Personnel

Instructor Office Hours: Monday 2-3pm (SN 255).
Assignments will be handled via Canvas..

Undergrad Learning Assistants
  • Andrea Dunn, asdunnbeltran@unc.edu
  • William Li, lsy0320@email.unc.edu
  • Liujie Zheng, liujiez@email.unc.edu
Office Hours: Thrs 4.30-5.30 (SN 252)

Goal/Student Learning Outcomes

(a) Understand the fundamentals of Computer Vision. (b) Develop implementation expertise by writing codes in Python to solve various Computer Vision tasks. (c) Develop strong mathematical frameworks required for problem solving. (d) Learn to read and write scientific articles and publications.

Course Policies

Grading (for both 590 & 790):

  • Assignment 1: [pen & paper] Linear Algebra and Probability Recap: 5%
  • Assignment 2: [pen & paper + coding in python] Frequency domain image analysis: 10%
  • Assignment 3: [coding in python] Stitching images to generate panorama: 10%
  • Assignment 4: [coding in Google Colab] Deep Learning 1: 15%
  • Assignment 5: [pen & paper] 3D Vision 1: 10%
  • Assignment 6: [pen & + coding in python] 3D Vision 2: 10%
  • Final-exam: [pen & paper] in-class + take-home: 30%
  • Class Participation & Quizzes: 10%

Grades for 590 students will be curved differently from 790 students.

Misc.

Final exam: Final exam will be a mixture of in-person & take-home. Students are expected to finish the final exam in-person in 75 minutes. However you can choose to finish some questions by taking it home and submitting it online by the end of the day. You will be graded for only 80% of the total points assigned for each question you submit as take-home.

Class Participation: Class participation points will be provided based on random quizzes throughout the class.

Late Submissions: Late assignments will not be accepted. You lose 5% of total points for that assignment for each late day. No exception allowed.

Academic Integrity & Collaboration:
  • For your assignments, you are allowed to use materials from external sources as long as it helps you to understand the topic and NOT actually solve the assignment problem.
  • You MUST NOT use homework solutions from previous years.
  • You MUST clearly acknowledge any sources used for solving the assignment.
  • You can discuss and brainstorm in groups but the programming and the solution has to be done on an individual basis.
  • No copying or replicating of other existing solutions.
  • Final-exam is in class. You can bring 1 US Letter size cheat-sheet (1-sided). But collaboration, electronic devices, browsing the web, printed materials etc. are not permitted.
  • ANY VIOLATION OF ACADEMIC INTEGRITY WILL BE REPORTED TO THE UNC OFFICE OF STUDENT CONDUCT

Schedule

Date Topic Details Special Dates
Intro & Review
Tues Aug 22 Intro. to Computer Vision & Ethical Concerns Lecture Slide
Thrs Aug 24 Maths Review (Linear Algebra, Probability, Calculus) Lecture Slide [pdf] HW1: Maths review (assigned)
Colors & Imaging
Tues Aug 29 No Class
Thrs Aug 31 Color & Color Spaces Lecture Slide [pdf]
Tues Sept 5 No Class (Well-being day) HW1: Maths review (due)
Thrs Sept 7 In-Camera Imaging Pipeline Lecture Slide [pdf]
Image Processing
Tues Sept 12 Filtering - Convolution, Gradients, & Edges Lecture Slide [pdf]
Thrs Sept 14 Frequency domain - Fourier Analysis Lecture Slide [pdf] HW2: Image Processing (assigned)
Features
Tues Sept 19 Feature Detection (Corner & Blob) Lecture Slide [pdf]
Thrs Sept 21 Feature Descriptor & Matching (SIFT) Lecture Slide [pdf]
2D Transformation
Tues Sept 26 2D Transformations & Fitting Lecture Slide [pdf]
Thrs Sept 28 RANSAC + Image Blending Lecture Slide [pdf] HW2: Image Processing (due)
HW3: Panorama (assigned)
Learning & Perception
Tues Oct 3 Recognition Lecture Slide [pdf]
Thrs Oct 5 Detection Lecture Slide [pdf]
Tues Oct 10 Segmentation & Matting Lecture Slide [pdf]
Thrs Oct 12 No Class (University Day) HW3: Panorama (due Friday Oct 13)
HW4: Deep Learning (assigned)
Tues Oct 17 Generative Models in Computer Vision Lecture Slide [pdf] Online Lecture
Thrs Oct 19 No Class (Fall Break)
3D Vision
Tues Oct 24 Camera Models + Calibration - 1 Lecture Slide [pdf]
Thrs Oct 26 Camera Models + Calibration - 2 Lecture Slide [pdf] HW4: Deep Learning (due)
HW5: 3D Vision 1 (assigned)
Tues Oct 31 Two-view Geometry-1 Lecture Slide [pdf]
Thrs Nov 2 Two-view Geometry-2 Lecture Slide [pdf]
Tues Nov 7 Stereo Lecture Slide [pdf]
Thrs Nov 9 Multi-view Stereo Lecture Slide [pdf] HW5: 3D Vision 1 (due)
HW6: 3D Vision 2 (assigned)
Tues Nov 14 Structure from Motion Lecture Slide [pdf]
Thrs Nov 16 Structure from Motion (cont.)a Lecture Slide [pdf]
Tues Nov 21 Light & Photometric Stereo Lecture Slide [pdf]
Thrs Nov 23 No Class (Thanksgiving)
Tues Nov 28 NeRFs Lecture Slide [pdf] HW6: 3D Vision 2 (due on Wed Nov 29)
Thrs Nov 30 Mid-term Review
Tues Dec 5 Final Exam (in-class and/or take home) Syllabus: Whole course