Structure from Motion with uncalibrated image sequences

One of our main research topics is the reconstruction of 3D-objects from uncalibrated image sequences. These image sequences can aquired by video cameras, photo cameras or digital cameras without the need of any camera calibration. The need for these real-world 3D-models is increasing constantly, driven by the extensive use of computer graphics in the game industry, Electronic Commerce and the new challenges in the special effects film industry. Production of photorealistic models of real scenes is still very time consuming and feasible only with expensive 3D-scanners in conjunction with manual interaction.

We present a method for automatic and photorealistic modeling of 3D-objects from image sequences, video sequences or mixed image and video sequences. The camera is moved freely around the object and used as automatic 3D-scanner. For that it is needed to automatically track the camera path and to estimate the camera viewing parameters. Then, the 3D-model is computed from the different views of the object and stored as VRML-object. The algorithmic steps of the method are:

1. Image capture with freely moving photo or video camera,
2. tracking of the camera path and estimation of camera parameters from images (papers),
3. depth estimation of 3D scene from images depending on camera path and parameters,
4. realistic 3D-object surface modelling of the scene objects (papers).

The method is multi-purpose for modelling of realistic 3D-objects in complex scenes. It covers the reconstruction of small detailed objects up to large building fronts and landscapes with a simple photo camera or a consumer video camera.

Examples for scene reconstruction from image sequences

We take five digital photos from the kiel water tower. Then we run the unclaibrated structure from motion algorithm and get the model.

Movie for presentation on CeBIT 2003

Movie 720x576 PAL (!! 100MB)

Movie 352x288 PAL (!! 37MB)

Flyer on CeBIT 2003

Flyer

Literature

M. Pollefeys, R. Koch and L. Van Gool. Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters, accepted for IJCV Special Marr Price Issue (International Journal of Computer Vision).

R. Koch: 3-D Modeling of Objects from Image Sequences. in: B. Jaehne, H. Haussecker, P. Geissler(eds.): Handbook of Computer Vision and Applications. Academic Press, 1999.

R. Koch, L. Van Gool (eds.): 3D Structure from Multiple Images of Large-Scale Environments. Proceedings of the SMILE workshop, Freiburg, Germany. LNCS 1506, Springer, June 1998.