In a recent alternative research path for interactive 3D graphics the
scene to be rendered is described with images. Image-based rendering (IBR)
is appealing since natural scenes, which are very difficult to model conventionally,
can be acquired semi-automatically using cameras and other devices. Also
IBR has the potential to create high-quality output images if the rendering
stage can preserve the quality of the reference images (photographs). One
promising IBR approach enhances the images with per-pixel depth. This allows
reprojecting or warping the color-and-depth samples from the reference
image to the desired image. Image-based rendering by warping (IBRW) is
the focus of this dissertation.
In IBRW, simply warping the samples does not suffice for high-quality
results because one must reconstruct the final image from the warped samples.
This dissertation introduces a new reconstruction algorithm for IBRW. This
new approach overcomes some of the major disadvantages of previous reconstruction
methods. Unlike the splatting methods, it guarantees surface continuity
and it also controls aliasing using a novel filtering method adapted to
forward mapping. Unlike the triangle-mesh method, it requires considerably
less computation per input sample, reducing the rasterization-setup cost
by a factor of four.
The algorithm can be efficiently implemented in hardware and this dissertation
presents the WarpEngine, a hardware architecture for IBRW. The WarpEngine
consists of several identical nodes that render the frame in a sort-first
fashion. Sub-regions of the depth images are processed in parallel in SIMD
fashion. The WarpEngine architecture promises to produce high-quality high-resolution
images at interactive rates.
This dissertation also analyzes the suitability of our approach for
polygon rendering and proposes a possible algorithm for triangles. Finding
the samples that must be warped to generate the current frame is another
very challenging problem in IBRW. The dissertation introduces the vacuum-buffer
sample-selection method, designed to ensure that the set of selected samples
covers every visible surface.