Roni Sengupta
Assistant Professor · Department of Computer Science · UNC Chapel Hill
Spatial & Physical Intelligence (SPIN) LabI lead the SPIN Lab at UNC Chapel Hill. My research lies at the intersection of Computer Vision and Computer Graphics, mainly centered around 3D Vision and Computational Photography.
My lab is particularly interested in developing AI techniques that can understand physics from images and videos by solving Inverse Physics problems — estimate shape, motion, physical properties (e.g., reflectance, deformation) and external conditions (e.g., lighting, force) from visual data. We solve Inverse Physics to advance various applications in Immersive Media (AR/VR, content creation, telepresence), Healthcare (spatial analytics and robotics-assisted surgery from endoscopy & laproscopy) and various scientific and engineering applications.
Prior to UNC, I was a Postdoctoral Research Associate at University of Washington, working with Prof. Steve Seitz, Prof. Brian Curless, and Prof. Ira Kemelmacher-Shlizerman in the UW Reality Lab / GRAIL (2019–2022). I completed my Ph.D. at University of Maryland – College Park (2013–2019), advised by Prof. David Jacobs, and my undergraduate degree in Electronics & Tele-Communication Engineering from Jadavpur University, Kolkata, India (2009–2013). I have also collaborated with researchers at NVIDIA Research, Snapchat Research, The Weizmann Institute of Science, and TU Dortmund.
Awards & Honors
Research
Our research sits at the intersection of Computer Vision and Computer Graphics, with a focus on 3D Vision and Computational Photography. We develop AI techniques that enable machines to understand the spatial and physical structure of the world from visual data.
Our lab is particularly interested in developing AI techniques that can understand physics from images and videos by solving Inverse Physics problems — estimate shape, motion, physical properties (e.g., reflectance, deformation) and external conditions (e.g., lighting, force) from visual data. We solve Inverse Physics to advance various applications in Immersive Media (AR/VR, content creation, telepresence), Healthcare (spatial analytics and robotics-assisted surgery from endoscopy & laproscopy) and various scientific and engineering applications.
Focused research themes:
Inverse Rendering
Inverse rendering seeks to recover the physical properties of a scene — geometry, material reflectance, and lighting — from images or videos. Our research combines physically-based rendering with learning-based models to build robust and generalizale inverse rendering techniques.
Relevant Publications
- GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures [ArXiv'25]
- GLOW: Global Illumination-Aware Inverse Rendering of Indoor Scenes Captured with Dynamic Co-Located Light & Camera [CVPR Findings'25]
- NFL-BA: Improving Endoscopic SLAM with Near-Field Light Bundle Adjustment [NeurIPS'25]
- MVPSNet: Fast Generalizable Multi-view Photometric Stereo [ICCV'23]
- SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild [CVPR'18]
Inverse Simulation
Inverse simulation involves recovering an object's 3D geometry and physical properties — such as material stiffness or initial force conditions. Our research combines physical simulation with learning-based models to build inverse simulation techniques for sparse-view and monocular videos.
Relevant Publications
Generative Physics
Generative Physics involves developing controllable generative models that can alter various physical attributes - lighting, deformation, aging, semantic attributes - in images and videos. Our research focuses on developing creative solutions for controllable editing without requiring large training data. Wefocus on developing personalized, training-free methods that resolve the long-standing trade-off between inversion accuracy and editability in generative image editing frameworks.
Relevant Publications
- HarmoVid: Relightful Video Portrait Harmonization [CVPR'26]
- Over++: Generative Video Compositing for Layer Interaction Effects [ArXiv'25]
- ScribbleLight: Single Image Indoor Relighting with Scribbles [CVPR'25]
- The Aging Multiverse: Generating Condition-Aware Facial Aging Tree via Training-Free Diffusion [SIGGRAPH Asia'25]
- MyTimeMachine: Personalized Facial Age Transformation [SIGGRAPH'25]
- Personalized Video Relighting With an At-Home Light Stage [ECCV'24]
Inverse Physics for Endoscopy
3D perception in endoscopy (colon, upper & lower airway) unlocks critical applications in medical imaging: automated measurement of organ geometry, enhanced visualization for diagnosis, and guidance for robotic surgery. The task is extremely challenging due to complex lighting effects (near-field illumination, global light transport, specular highlights, and subsurface scattering) and tissue deformation (due to breathing, endoscope insertion, surgery). We form a collaborative research team with experts in medical imaging, robotics, gastroenterology, pulmonology, and otolaryngology.
Relevant Publications
- Stop Holding Your Breath: CT-Informed Gaussian Splatting for Dynamic Bronchoscopy [ArXiv'26]
- Understanding Model Behavior in Monocular Polyp Sizing [ArXiv'26]
- NFL-BA: Improving Endoscopic SLAM with Near-Field Light Bundle Adjustment [NeurIPS'25]
- Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos [ECCV'24]
- Structure-preserving Image Translation for Depth Estimation in Colonoscopy [MICCAI'24]
We are grateful for the generous support from our sponsors.
Publications
Team
PhD Students
Research Engineer
We are looking for strong research focused MS and BS students. If you are already a student at UNC, and interested in joining SPIN lab, please reach to me via email.
MS Students
Undergraduate Students
Alumni
Former PhD Students
Former MS Students
Former Undergraduates
- Amisha Wadhwa → Bank of America
- Pierre-Nicolas Perrin (BS–MS) → Capital One
- Yulu Pan (BS) → PhD at UNC
- Max Christman (BS) → MS at UNC
- Andrey Ryabstev (BS–MS, UW) → Google
- Peter Lin (BS–MS, UW) → ByteDance
- Jackson Stokes (BS–MS, UW) → Google
- Peter Michael (BS–MS, UW) → PhD at Cornell University
Teaching
- COMP 590 Introduction to Computer Vision (Undergraduate focused) Fall 2024 Fall 2025 Fall 2026
- COMP 776/590 Computer Vision in 3D World (Graduate focused) Spring 2023 Fall 2023 Spring 2025 Spring 2026
- COMP 790 3D Generative Model Spring 2024
- COMP 790/590 Neural Rendering Fall 2022
Lab Photos
