The Ultrasound Project
The Ultrasound Project
We are exploring the application of augmented reality systems to the problems of medical visualization. Specifically we are using ultrasound echography imaging, a video see-through head-mounted display, and a high performance computer graphics engine to allow a physician to see within a patient. We want to give the user Superman's "X-ray eyes" into the body.
We have chosen ultrasound as our imaging modality because it is non-invasive, it does not involve harmful radiation, and it is fast. We can acquire new images at video rates (30 Hz). On the down side the quality of ultrasound images is poor compared to X-ray CT or MRI.
Our second system attempted to improve the visualization through volume rendering. In this system we acquired multiple ultrasound slices, reconstructed these slices into a rectilinear volume data set, and rendered the data in real-time on Pixel-Planes 5. Slices were added to the volume at a rate of ~1 Hz, and images were rendered at ~10 Hz. Even though we had a newer ultrasound machine, better tracking (with an Ascension), and better rendering, the images were disappointing.
This image shows what the user saw in our second system. As the physician sweeps the transducer over the patient's belly, slices are sampled into the the volume. Note the wire frame representations of the volume, the fan-shaped slice, and the transducer.
Here is a link to a discussion on Calibration in our research group. The picture at the top of this document shows our visualization of what a user would see looking through a HMD. Click Me for more discussion on our system and its results.
Our goal for image quality has always been to generate an image that a child would see and recognize a baby within a mother. Our best results have not quite achieved this goal. One of the ways we are trying to improving our images is to acquire a better ultrasound machine. Most machines, ours included (generously donated by PIE Medical), acquire slice images that have a certain thickness, typically on the order of 1-2 cm. This is fine for viewing one slice, but when multiple slices are put together, the volume is blurry in the direction perpendicular to the slice. Essentially, the resolution of ultrasound in that direction is low. Therefore we, with Dr. Ricardo Hahn, are trying to acquire a machine with thin slice transducers.
In order to go back to a real-time interactive system, we are considering building a system similar to our first one, where individual slices are display in the virtual environment. However this time we will us our SGI Reality Engine to perform the rendering, use our optical tracking system, and our improved HMD calibration methods. Pixel-Planes 5 was not designed to render image textures, so rendering slices is difficult. We had to make each pixel in each image a small polygon. Therefore the system was very slow with more than a few slices. For the Reality Engine rendering a textured polygon is no problem, therefore the system should be much more responsive.
Dave Chen (chen@cs.unc.edu)