In this section our goal is to create a lightfield model from a scene to render new views interactively. Our approach has been presented in a number of consecutive papers [68,67,56]. For rendering new views two major concepts are known in literature. The first one is the geometry based concept. The scene geometry is reconstructed from a stream of images and a single texture is synthesized which is mapped onto this geometry. For this approach, a limited set of camera views is sufficient, but specular effects can not be handled appropriately. This approach has been discussed extensively in this text. The second major concept is image-based rendering. This approach models the scene as a collection of views all around the scene without an exact geometrical representation . New (virtual) views are rendered from the recorded ones by interpolation in real-time. Optionally approximate geometrical information can be used to improve the results . Here we concentrate on this second approach. Up to now, the known scene representation has a fixed regular structure. If the source is an image stream taken with a hand-held camera, this regular structure has to be resampled. Our goal is to use the recorded images themselve as scene representation and to directly render new views from them. Geometrical information is considered as far as it is known and as detailed as the time for rendering allows. The approach is designed such, that the operations consist of projective mappings only which can efficiently be performed by the graphics hardware (this comes very close to the approach described in ). For each of these scene modeling techniques the camera parameters for the original views are supposed to be known. We retrieve them by applying known structure and motion techniques as described in the previous chapters. Local depth maps are calculated applying stereo techniques on rectified image pairs as previously explained.