COMP 790-177 Special Topics: 3D Generative Model

Spring 2024

Instructor: Roni Sengupta
Tuesday and Thursday, 2:00-3:15pm, FB 0007

[ Overview | Course Details | Schedule | Pre-Requisite ]

Course Description

This is an advanced seminar course focusing on recent developements in the field of 3D Generative Model. This course will loosely focus on understanding how generative models can be useful in reconstructing, synthesizing and editing 3D shapes or 3D-aware 2D images from various multimodal inputs, e.g. text, sketch, RGB or RGBD images.The course will specifically focus on latest topics centering text-to-3D, diffusion models, and gaussian splatting. At the beginning of the course, the instructor will present introductory lectures on generative models and 3D representations, which will be followed be detailed paper discussions on various sub-topics. This course will also involve working on a research topic centered around 3D generative model.

Office Hours: By appointment only (SN 255).

Target Audience

790: PhD students focusing on latest research trends in Computer Vision & Graphics

Goal/Student Learning Outcomes

790: (a) Understand details of different recent techniques in 3D generative model (b) Formulate novel project ideas and explore new solutions. (c) Improve paper and project presentation skills.

Grading

  • 2-3 Paper Presentation (exact number TBD): 20% grade
  • 5-10 Paper review & discussion (exact number TBD): 30% grade
  • Course Project: 50% grade (pitch: 10%, update-1: 10%, update-2: 10%, final: 20%)

Paper Presentation

Each paper will be given a 20 minutes presentation slot, followed by 10 mins of Q&A. Present the paper as if you are the lead author. Please follow Prof. Gedas Bertasius's instructions on paper presentation. Your grades will depend on how well you present the paper and how well you answer the questions and clarify doubts from audience. You will also upload the presentation slide deck by 11.59pm on the day of the presenatation.

Paper Review & Discussion

Write a short review that should include: (a) summarize the paper in 5-6 sentences; (b) list what you consider strength of the paper; (c) list potential drawbacks of the paper; (d) Atleast 1 question that you would like the presenter and the rest of the class to discuss. Paper review is due 11.59pm on the day BEFORE the presentation of the paper. After the paper presentation each reviewer will ask their question for discussion. Fill your paper review using this google form.

Course Projects

Goal: Goal of the course project is to write a research paper that can be submitted to Neurips, Siggraph Asia, AAAI, CVPR or other conferences. The topic of the paper should involve 3D generative models.

Team: The course encourage students from slightly different research backgrounds to form a team and explore new exciting research projects. Ideal group size is 2. Group size of 1 or 3 can be permitted under special circumstances. You will need to clarify if the proposed project is a part of the research project you are working with your advisor or not. Expectations and grades will be scaled accordingly.

Project pitch: 5 mins presentation about your project. You should clearly present: (a)The problem that you are trying to solve. (b) The motivation behind the problem. (c) Your proposed approach, and how it relates to prior work. (c) The experiments that you plan to conduct. (d) Your plan for evaluation. You will need to upload your slides.

Project updates 1 & 2: 10 mins presentation. You should extend your project proposal to include the following: (a) A preliminary set of results. (b) Your analysis of those results. (c) Additional experiments that you plan to run. You will need to upload your slides.

Final: 15 mins presentation (due May 3) + final paper in CVPR/Sigasia/Neurips/AAAI format, 8 page (due May 10).

Feel free to email me and set up a meeting to discuss your project ideas.

Misc.

Late Submissions: The class is structured around a tight paper presentation schedule. Therefore, late assignments will not be accepted. If you have other deadlines you can always reschedule your paper presentation ahead of time.

Academic Integrity: For your presentations and projects, you are allowed to use materials from external sources. However, you must clearly acknowledge those sources.

Exact paper names can be updated given this is an evolving field and new papers are coming out everyday. Exact paper names will be finalized 2 weeks before the presentation.

Schedule