In a recent alternative research path for interactive 3D graphics the
scene to be rendered is described with images. Image-based rendering (IBR)
is appealing since natural scenes, which are very difficult to model conventionally,
can be acquired semi-automatically using cameras and other devices. Also
IBR has the potential to create high-quality output images if the rendering
stage can preserve the quality of the reference images (photographs). One
promising IBR approach enhances the images with per-pixel depth. This allows
reprojecting or warping the color-and-depth samples from the reference
image to the desired image. Image-based rendering by warping (IBRW) is
the focus of this dissertation.
In IBRW, simply warping the samples does not suffice for high-quality results because one must reconstruct the final image from the warped samples. This dissertation introduces a new reconstruction algorithm for IBRW. This new approach overcomes some of the major disadvantages of previous reconstruction methods. Unlike the splatting methods, it guarantees surface continuity and it also controls aliasing using a novel filtering method adapted to forward mapping. Unlike the triangle-mesh method, it requires considerably less computation per input sample, reducing the rasterization-setup cost by a factor of four.
The algorithm can be efficiently implemented in hardware and this dissertation presents the WarpEngine, a hardware architecture for IBRW. The WarpEngine consists of several identical nodes that render the frame in a sort-first fashion. Sub-regions of the depth images are processed in parallel in SIMD fashion. The WarpEngine architecture promises to produce high-quality high-resolution images at interactive rates.
This dissertation also analyzes the suitability of our approach for polygon rendering and proposes a possible algorithm for triangles. Finding the samples that must be warped to generate the current frame is another very challenging problem in IBRW. The dissertation introduces the vacuum-buffer sample-selection method, designed to ensure that the set of selected samples covers every visible surface.