CAREER: Visual 3D Acquisition, Modeling and Rendering of the Real World

Principal Investigator: Marc Pollefeys 
Funding Agency: National Science Foundation 
Agency Number: IIL-0237533

Abstract

In recent years tremendous progress has been made towards recovering 3D models of objects and scenes from images of the real world. It is now possible to walk around an object with a photo- or video camera and let a computer automatically construct a realistic visual representation from the recorded imagery. However, many real world scenes are still too hard for todays algorithms. Non-rigid motions and complex reflectance properties can make accurate reconstruction very difficult. Approaches to efficiently capture complete environments are also still lacking. In addition, for many applications it is not sufficient to re-render the scene or object as it was recorded, but modifications to lighting or other properties might be required. The long term goal of this research problem is to tackle these problems. We plan to develop automatic visual 3D modeling approaches that can deal with objects, people and complete environments, including complex shapes, reflectances and non-rigid motions. The resulting models should make it possible to render new images, possibly with different lighting, and to perform 3D measurements, opening up a wide range of applications. An important aspect will consist of also recovering camera motion and calibration during acquisition.