next up previous contents
Next: Rendering from recorded images Up: Lightfield model Previous: structure and motion   Contents

Lightfield modeling and rendering

In [87] the appearance of a scene is described through all light rays (2D) that are emitted from every 3D scene point, generating a 5D radiance function. Subsequently two equivalent realizations of the plenoptic function were proposed in form of the lightfield [78], and the lumigraph [39]. They handle the case when the observer and the scene can be separated by a surface. Hence the plenoptic function is reduced to four dimensions. The radiance is represented as a function of light rays passing through the separating surface. To create such a plenoptic model for real scenes, a large number of views is taken. These views can be considered as a collection of light rays with according color values. They are discrete samples of the plenoptic function. The light rays which are not represented have to be interpolated from recorded ones considering additional information on physical restrictions. Often real objects are supposed to be lambertian, meaning that one point of the object has the same radiance value in all possible directions. This implies that two viewing rays have the same color value, if they intersect at a surface point. If specular effects occur, this is not true any more. Two viewing rays then have similar color values if their direction is similar and if their point of intersection is near the real scene point which originates their color value. To render a new view we suppose to have a virtual camera looking at the scene. We determine those viewing rays which are nearest to those of this camera. The nearer a ray is to a given ray, the greater is its support to the color value.

The original 4D lightfield [78] data structure employs a two-plane parameterization. Each light ray passes through two parallel planes with plane coordinates $(s,t)$ and $(u,v)$. The $(u,v)$-plane is the viewpoint plane in which all camera focal points are placed on regular grid points. The $(s,t)$-plane is the focal plane. New views can be rendered by intersecting each viewing ray of a virtual camera with the two planes at $(s,t,u,v)$. The resulting radiance is a look-up into the regular grid. For rays passing in between the $(s,t)$ and $(u,v)$ grid coordinates an interpolation is applied that will degrade the rendering quality depending on the scene geometry. In fact, the lightfield contains an implicit geometrical assumption, i.e. the scene geometry is planar and coincides with the focal plane. Deviation of the scene geometry from the focal plane causes image degradation (i.e. blurring or ghosting). To use hand-held camera images, the solution proposed in [39] consists of rebinning the images to the regular grid. The disadvantage of this rebinning step is that the interpolated regular structure already contains inconsistencies and ghosting artifacts because of errors in the scantily approximated geometry. During rendering the effect of ghosting artifacts is repeated so duplicate ghosting effects occur.



Subsections
next up previous contents
Next: Rendering from recorded images Up: Lightfield model Previous: structure and motion   Contents
Marc Pollefeys 2002-11-22