Go to final report web page.

The idea:

Picture this - there is a volume of interest that we want to track activity in. Whenever a foreign object (say the user's hand) enters this region, its shape is determined and it is tracked using only a camera and a projector projecting structured light patterns. So we are able to keep track of the object as it moves and project something on it, without needing to attach a tracker to it or use any other form of tracking.

If this vision (or the project title) sounds grand, it is understandably so. This is to be the first major result of my research in the STC Office of the Future group. It is unrealistic to expect that this will be possible in this semester. However, as a preliminary result I hope to have localization done for a static object (depth extraction or 3D photography) so that the system knows the 3D shape of the object and can reliably project a marker (say a line or cross) on the "tip" of the object as defined by the distance from a convenient reference plane.

Tasks:

• Find a camera and projector pair hooked up to PCs or SGI.
• Sort out issues of which system to work on - PC or SGI. Both have problems. Currently the PCs we have don't have graphics cards with support for multiple monitors. So maybe two PCs have to be networked - one controlling the projector (server) and the other the camera (client). SGI (evans) on the other hand can have availability problems later in the semester, is not strictly needed, and the cameras / projectors that I currently use that are hooked up to evans are not the best (camera has high distortion and low dynamic range, projector is mounted too far away and produces low signal strength in the structured light, which makes it difficult to detect in the camera view.
• I have already written an SGI program for projection and camera, port it to PC if required.
• Calibration of the projector and camera to the same coordinate system.
• Tabulation of a function which gives the 3D coordinates of a point given the current position of the structured light feature (say a vertical line) and the (x,y) coordinates in the camera image where it shows up.
• Doing a table lookup in real time (if possible) to determine if we are intersecting with any object that lies within the region of interest.
• Tracking the interesting object (in its simplest form, this works out to just moving a vertical line such that it hovers on the border between object and no object, defining the "tip" of the object.
• Probably a ton others will come up...

Deliverables:

I hope to give a live demo of the system, whatever progress I have made working on it, and any results.