Augmented Reality Scientists Want Respect

by Miguel Helft
5:03am  2.Jun.97.PDT Soon, goggle-wearing surgeons at the University of North Carolina may be able to see "through" a human body as they perform a biopsy. Similarly, researchers at Columbia University are working on a system that would let a contractor view the structure of a building "through" its walls. And at the University of Rochester, Jim Vallino is developing a technology that would give soldiers the ability to see hidden enemy troop movements, relayed from a reconnaissance airplane, in the battlefield in front of them.

These are among a dozen or so places around the world where scientists are doing research in augmented reality. Although overshadowed by its popular relative, virtual reality, scientists believe that augmented reality could soon yield an array of useful applications ranging from manufacturing to medicine to the much-touted "wearable" computers of the future.

The key difference between the two technologies is that while virtual reality creates a completely illusory environment, augmented reality supplements the real world -- through a head-mounted display, for example -- with information intended to make a particular task easier.

"In virtual reality, the user is pretty much immersed in a computer-generated world. The physical world is left behind," said Vallino. "In augmented reality you keep the user in the real world, but you add some virtual objects into the person's view of the world."

The potential for applications is promising and the field is gathering momentum. An upcoming issue of Presence, the leading academic journal dedicated to virtual environments published at MIT, will be entirely focused on augmented reality research.

Already, Boeing has developed augmented-reality circuit-layout diagrams that are superimposed on a real circuit board for manufacturing. Likewise, Rajeev Sharma, an assistant professor of computer science at Pennsylvania State University, developed an assembly system that tracks 15 to 20 parts. Using a database, the system creates a 3-D diagram that shows the user exactly how to assemble the parts.

"Eventually, augmented reality is going to become more important than virtual reality," Sharma says. "People are beginning to see its benefits, but it hasn't fully caught on because of the technological barrier."

Indeed, augmented reality faces a number of technical hurdles. The most significant, which scientists call "registration," is the ability to superimpose the computer graphics on the real world so that they coexist seamlessly.

"When you merge the real world and the virtual world, it has to look as if they belong together," Vallino says. "As you move around, the computer graphics have to be told how to re-render the virtual scene."

This is difficult because human eye is extremely sensitive to even the most minute misalignment between the two worlds. At the same time, errors that are barely noticeable in virtual reality are big problems in augmented reality -- clearly, a surgeon who is off target by even the slightest margin could cause irreparable harm.

And just like in virtual reality, augmented reality tracks the movements of the user and re-renders the virtual scene accordingly. But in augmented reality, the system also needs to track the movement of real objects in the scene - a process that is very difficult to achieve.

"You need to know where to display the virtual world, when to display it, and what to display," Sharma says.

Placing colored dots on a variety of objects, Vallino's research focuses on the tracking problem using a hand-held video camera. The dots serve as points that give the computer system a frame of reference on which the virtual world is rendered. As they move, the video camera transmits the new position to the computer and the virtual world is re-drawn accordingly.

Eventually, augmented reality will completely transform the way humans and computers interact, says Ulrich Neumann, an assistant professor of computer science at the University of Southern California.

"Now you have whatever a 14-inch screen has to offer," Neumann says. "When you put on a head-mounted display, and you'll be immersed in a real 3-D world that is data and information permeated." In other words, the augmented-reality world is like the real world but adorned with useful computer-generated Post-Its, he says.

Neumann cautions that augmented reality isn't going to happen overnight. Limited applications with fairly static worlds - and therefore easier tracking and registration problems - will come first. But before it catches on, augmented-reality researchers are going to have to come up with a widely used application in the business or manufacturing world.

"It's going to take some creative people, creative thoughts, and creative work to translate even the existing technology into a killer application," Neumann says.


Copyright © 1993-97 Wired Ventures, Inc.
Compilation copyright © 1997 HotWired, Inc.
All rights reserved.

This article is copyrighted by Wired Ventures, Inc. and may be redistributed provided that the article remains intact, with this copyright message clearly visible. Under any circumstances, this article may not be re-sold or re-distributed for compensation of any kind without prior written permission from Wired Ventures, Inc.