Main Idea

We are used to looking at the output of a computer graphics program on a monitor or on a screen. But, how can we see the "special effects" directly in our environment ? For example, how can we make a clay vase sitting on a table look like it is made up of gold, with fine details ? The appearance of an object is a function of illumination, surface reflectance and viewer location. Hence, we can rearrange the factors along the optical path and reproduce the equivalent desired appearance.

We have recently introduced a new paradigm and related techniques to graphically animate physical objects with projectors. Because the approach is to effectively "lift" the visual properties of the object into the projector, we call the projectors shader lamps. We address the central issue of complete illumination of non-trivial physical objects using multiple projectors and present a set of new techniques that make the process of illumination practical. We show some results and describe the new challenges in graphics, geometry, vision and user interfaces.

FAQ

What is Shader Lamps ?
In the simplest form, Shader Lamps is a technique to change the appearance of physical objects. For example, you can make a white clay object sitting on a table in front of you appear to be made of gold or plastic. The physical object is illuminated with a data (or video or slide) projector. The images to be projected are computed with a 3D graphics rendering program.

What is the main idea ?
You see special effects typically on a movie screen or on TV. By changing appearance of real objects, Shader Lamps achieves 'special effects in the real world' around you.

What is required to achieve the Shader Lamps effects ?
A physical object, 3D graphics model of the same object, a projector and a PC running the rendering program.

How are the projected images aligned with real objects ? Is this process cumbersome ?
The registration involves two stages.

During the calibration stage, locations of projector pixels that illuminate important features on the physical object are collected. The calibration computes the pose of the projector with respect to the physical object. This one-time pre-processing stage involves the simple process of clicking with a mouse to collect pixel locations and takes less than one minute !

During the rendering stage, the 3D graphics model is rendered from the viewpoint of the projector. The rendered image when displayed appears correctly aligned with important features on the physical object.

How are the images generated ?
Image can be generated on the fly using real time 3D graphics rendering and animation. The images or image-sequences can also be precomputed and displayed one at a time by the projector.

What type of objects can be illuminated ?
Objects may be planar or non-planar. They could be on a tabletop, room-sized or buildings. Typically neutral colored objects (white or light grey with diffuse matte finish) are well suited.

How can you avoid self-shadows for complex objects (like the Taj Mahal model) ?
For a non-trivial object, shadows wrt a single projector are unavoidable. Shadows cast by the object on itself can be 'filled in' by two or more projectors. Multiple projectors are also required to surround and achieve complete illumination of a closed object.

How can you merge images from multiple projectors ?
A new intensity blending algorithm for feathering and cross-fading has been developed. It works in the presence of non-contiguous overlaps and surface shape discontinuities.

Can you illuminate objects that may be moving ?
If the rigid transformation of the moving object is known (e.g. if the object is on a turntable), it is straightforward to apply the transformation of the graphics model so that the physical and graphics model are back in the same coordinates, synchronized.

Can you make static objects appear as if they are moving ?
In addition to change of appearance, Shader Lamps techniques are ideal for creating interesting illusions. In some limited cases, by projecting appropriate animation physical objects can be made to appear as if they are moving. Some examples include a rotating vase and moving car.

At shaderlamps.com you can see more examples.