Being There

WHAT:
a new projector-based approach to visualizing re-creations of real and imaginary sites.

GOAL:
to achieve re-creations that are both visually and spatially realistic¡ªto provide users a strong sense of immersion as they walk around the virtual site.

BASIC IDEA:
use simple white building blocks to build static physical model that approximates the geometry of the site, and project dynamic imagery onto the blocks.


We envision a ¡°sea of projectors¡± in a warehouse-sized interior, in which we can re-create most indoor scenes.
ADVANTAGES:
(1) very wide field of view, (2) real walking around, (3) reduced sensitivity to tracking errors and system latencies, (4) auto-stereoscopic vision, (5) natural addition of augmented virtuality, (6) provision of haptics.
First, we need the scene model. Here, we have a synthetic model created using a modeling software. For model of real places, we can use a range-scanner to acquire the geometric information and a camera to capture the colors.
We then create a display-surface model by using one of our own software tools to arrange some primitive blocks near to the surfaces of the scene model that we want to display. The block size is the same as that of the physical Styrofoam blocks that we are using in this experiment..
The physical display surfaces are constructed in a laboratory, according to the display-surface model, using industrial-purpose Styrofoam blocks. These Styrofoam blocks were kindly sponsored by Reddi-Form, Inc.
We use 6 projectors and two 3rdTech HiBall trackers for this experiment. The 6 projectors are first calibrated with respect to a common tracker's frame of reference.

After the projectors are calibrated, a wireframe of the display-surface model is projector onto the Styrofoam blocks. We use this method to fine-tune and verify the arrangement of the physical blocks.
Since the images on the display surfaces should be view-dependent, we need to track the user's eye position. A two-pass rendering approach, using projective texture mapping, is used to generate the correct imagery for the user.
The user can also interact with the virtual environment. Here, we use the second HiBall tracker to track the position of a virtual spray-brush. We can see from the photograph that the user is spray-painting on the moulding of a window that is physically not there.

This example shows the usefulness of being able to move physically around virtual objects, which gives the user a natural way to interact spatially with the virtual environment.