Our Current Ultrasound Work

Our most recent work in ultrasound is a real-time acquisition, off-line visualization experiment. So all the data is collected in real-time, as it would be if we had a fully real-time system. However the data massaging and visualization is performed off line to create the visualization. In the future we hope to do everything in real-time, but currently we are limited by computer technology.

The major steps in our current experiment come in three parts: calibration, data acquisition, and image generation. There is a separate page for a discussion of the calibration portion.

Data Acquisition

The data acquisition portion of the experiment is performed in real-time, with a live subject. This typically takes an hour with a mother and an obstetrician. The pieces of data we must gather are the HMD tracking data and HMD video, the ultrasound tracking data and ultrasound video, and the belly surface geometry.

The HMD and ultrasound tracking data is taken from our ceiling tracker, and stored to disk. The HMD and ultrasound video streams are stored on D2 digital tape. Note that currently we cannot track two things at a time with our ceiling tracker, so first the HMD data was acquired, then the ultrasound data was.

The belly's surface geometry is collected as a set of points. To find the points, we use the ultrasound transducer as a pointing device.

Image Generation

Image generation is the process of taking all the tracking and video image data we've collected, and massaging it to generate a HMD view. The ultrasound tracking and images are used to generate a volume of the patient's innards. The HMD tracking data is used to render from the user's viewpoint. The HMD video camera data is used to show the real environment in the virtual rendering. The belly surface geometry is used to create the pit metaphor in the visualization.

All the tracker data collected is filter with a non-causal low-pass filter. For the ultrasound transducer tracking at cut off frequency of 1 Hz is used. For the HMD tracking the cut off frequency we use is 6 Hz.

To collect the video images, first they were stored on D2 digital tape. The they were grabbed from the video tape using our VME frame grabber.

The ultrasound images can be arbitrarily oriented and position in space. Thus for each ultrasound video image, we have a tracker record of its position and orientation. Then we take all the images and reconstruct them into a regularly gridded volume. The reconstruction algorithm is described in Ryutarou Ohbuchi's dissertation.

The HMD video data is used to generate the see-through aspects of the virtual environment. The HMD tracking data is used to match the virtual viewpoint with the HMD camera's viewpoint. Thus the virtual data appears combined with the real world through the video camera.

The metaphor we use for the visualization in our augmented reality system is of a pit. So we simulate a pit within the belly of the patient. Previous experiments simply overlayed the virtual display over the video camera. This proved to be a very artifical looking display. Therefore we created the pit to give the HMD wearer more 3D visual cues in the display. To properly define the pit, we use the shape and position of the belly.

The computer graphics images are rendered using the vol2 volume render on Pixel Planes 5. Then they are combined with the video camera images using a video chroma-keyer.