Imagine a world where all the objects around you can be animated and
augmented interactively in real time; where you can, for instance, paint
virtual designs on objects in the environment, which then stay in place as
you modify or move them around!  This opens up new possibilities for
interaction in augmented environments, and gives rise to new applications
in tele-immersion, medicine, architecture, art and user interfaces.  We
have taken the first steps towards this ambitious goal, building a
prototype system for virtual painting on real movable objects.

Earlier we developed Shader Lamps, a new paradigm in projector-based
graphics, where multiple projectors are used to render a virtual object
that coincides with a physical object used as a display surface.  Our
approach "lifts" the lighting and material properties of the real object
into the model being projected, and can be used to animate the surface of
the object or make it appear to be made of some other material.  Our
present system, based on Dynamic Shader Lamps, allows virtual paint and 
textures to be applied to movable real objects simply by direct physical
manipulation of the object and a tracked stylus (paint brush).