Converting 2D Video to 3D with Applications to 3D-TV, Video Analysis and Compression

Principal Investigator: Marc Pollefeys 
Funding Agency: National Science Foundation
Agency Number: IIS-0313047

Abstract

The aim of this project is to augment traditional 2D video content with 3D depth using computer vision approaches. To achieve this goal the camera motion and settings will also have to be recovered from the video data. The main application that is envisioned is to convert existing 2D video content to 3D video to provide sufficient content for stereoscopic displays and enable applications such as 3D TV to emerge and flourish. Besides this, the intended research also has important applications in the context of video analysis and compression. We intend to develop reliable fully automatic approaches. Since it will not always be possible to compute the depth from the available image content (e.g. fixed camera), we intend to characterize those ambiguity and provide perceptually acceptable results (e.g. fade depth out when it can't be computed anymore). Given a 2D video stream, we intend to (1) compute the relative motion between the scene and the camera for each shot, (2) detect independent moving objects and compute their motion and deformation, (3) compute a detailed depth representation for each video frame.