Plenoptic Modeling:
An Image-Based Rendering System
Leonard McMillan
and Gary Bishop
Image-based rendering systems are a powerful new approach for generating real-time photorealistic computer graphics. They can provide convincing animations without an explicit geometric representation. The "plenoptic function" of Adelson and Bergen is a parameterized function for describing everything that is visible from a given point in space. We use this plenoptic function to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation. We also present our own image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function. In addition, we introduce a novel visible surface algorithm and a geometric invariant for cylindrical projections that is equivalent the epipolar constraint defined for planar projections.
Figure 1. A panoramic image constructed by registering
and reprojecting 36 images that were acquired by an uncalibrated video
camera as it was panned through 360 degrees.
Figure 2. A second panoramic image generated by registering
31 images acquired from a position approximately 60 inches from the first
panoramic image.
Figure 3. A display of the cylindrical epipolar curves
relating the two panoramic views. The two convergence points, called
epipoles, correspond to the virtual image of the second camera as seen
from the first, and the vanishing point along the optical axis of the
second camera.
Figure 4.Arbitrary reprojections of the first
panoramic image. Each image corresponds to a change in viewing
position. These images were generated by an parameterized
image-warp that directly maps pixels from the source image
to their position in the desired view. These reprojections are
accomplished without any explicit underlying geometric scene
description. Notice how both occlusion and perspective effects
are modeled correctly.
Figure 5. Just for fun here is a short
MPEG animation of 16 cylinders (635K).
Notice how these cylinders have a considerably larger
vertical field-of-view (~110 degrees) than the ones shown previously.
That's because these days we're acquiring images using a
fisheye lens. This requires adding a lens distortion model
to the registration process described in the paper.
Surprisingly, a fisheye lens is easier (only a single
parameter distortion model is needed) to undistort than
alternative wide-angle lenses. More details will be available
soon. You might also notice that our motorized translation
platform is visible along the bottom of these images. We'll
add more about that in the near future, also.
Figure 6. These frames are from a video that
was shown at SIGGRAPH'95.
If you'd like to actually see the animation for yourself
an MPEG version is available.
It's a little over 10 Mbytes long, runs about 5 minutes,
and was digitized at 5 fps.
We owe a lot (of praise and headaches) to two undergrads,
Steve Wendel and
Arthur Gregory,
who spent a large chunk of their summer working on this video.
For more about visibility. For more about warping.