UNC-CH Computer Science Educational Outreach 

In the late 1960’s, Ivan Sutherland wrote the seminal paper "The Ultimate Display" in which he outlined the research agenda for most of computer graphics for the next three decades. Sutherland’s work kicked off research in an area we now call virtual reality. In 1981, Jaron Lanier, an early pioneer of VR, coined the term virtual reality. What made VR so exciting and powerful to researchers like Sutherland and Lanier was its ability to make you believe that the virtual environment was real by combining traditional computer simulators with immersive displays. Thus, as the viewer and participant in the virtual world, you would be willing to suspend your disbelief and accept that the components within the virtual world were truly viable objects and actions.

Let’s take a look now at how VR gets you to suspend your disbelief. A VR system makes artificial reality look so real in three ways:

1. Visual Display
2. Tracking
3. Rendering

These three elements are the main components of a VR system.

Visual Display

When we want to step into a virtual environment we usually do so by putting on a head-mounted computer display (HMD). The HMD acts as your eyes and ears when you’re in the virtual environment by showing the computer world on two liquid crystal display (LCD) screens. You’ve probably seen a LCD screen on a Sony Walkman television. Instead of one screen like on a Sony Walkman television, there are two LCD screens in the head-mounted display. Each screen shows the virtual environment from a slightly different viewpoint, just like your eyes view the real world. To see this difference in your visual points of view, hold one finger up and extend your arm out directly in front of you. Close one eye, then the other. When you do this, your finger appears to move. That’s because your eyes are spaced slightly apart and they see the world from two slightly different vantage points. Each eye is sending information about its unique viewpoint to the visual cortex in your brain. There it’s recombined and interpreted as a 3-dimensional view. Likewise, there are two LCD screens in the HMD so that your eyes will still have two slightly different visual sources of information, making it possible for you to see the virtual world in 3-D.

As you can see, displaying the virtual environment is a very complicated component of a VR system. Moreover, as a result of its sophisticated mimicking of human sight, the display device masks its mechanical nature and creates an illusion of reality.

Once we’re comfortably immersed in the virtual environment, we need a way to move around. You can traverse distances in two ways: by moving your feet and walking or by pushing a button on a hand-held device, like a wand, and "flying." Let’s talk about what’s involved in moving in the virtual world.

There are several problems that we need to overcome. First, the computer needs to know where you are in the virtual world; it does so by using a tracking system to monitor your movements. We accomplish this by attaching a device to the top of the HMD and to the wand. The device emits a signal that is picked up by the computer’s sensor. In turn, the computer interprets the data from the sensor to calculate your head and body position in the virtual environment. With this information the computer changes the pictures or graphics of the virtual world to correspond with your position as you turn your head from side to side or move forward. By tracking your movements, the computer knows to redraw an object to appear closer to you as you move towards it. In turn, you have a greater sense that this artificial world is real.

The tracking system must include not only the HMD, but the wand as well. The wand functions like the HMD by signaling to the computer what you want the computer to do. In the virtual world, it is an extension of you, allowing you to pick up and move objects, or to propel yourself through space. Like the HMD, the wand also has a tracking device embedded within its structure that communicates the position and actions of your hand to the computer. As a result, you and the computer have an interactive relationship in which your movements or commands are instantly acted upon by the computer. Because your actions have a direct impact and effect on the virtual environment, you have a compelling sense of actually being in the virtual world.

When you put on a head-mounted computer display, the first things you notice are the pictures, or graphics, as computer scientists call them. It takes a lot of the computer’s power to draw the graphics that you see.

Computer graphics are made up of hundreds of dots of lights called pixels. Computer scientists use shapes like triangles and rectangles to organize the pixels and create a graphic. If you have studied geometry and algebra, you are learning the building blocks of computer graphics.

Though there are many ways to create computer graphics, one way is to use simple geometric shapes to represent objects. For example, by positioning three rectangles together, you can represent the frame of a table. To make the table look more realistic, computer scientists add textures and colors to the wire-framed shapes.

There are still many steps between the newly created graphics and an interactive virtual environment that are too complex to discuss here (see the bibliography at the end for further information). However, once the virtual environment is completed, there still remains the problem of redrawing this world as you look or move around it. Unlike a video game, we need powerful computing capabilities to draw or render images that correspond to your movements within the virtual world. When the computer draws or redraws a graphic, we call this action "rendering." In order to avoid a delay between your actions and the effect in the computer world, the computer must work very quickly to redraw all the objects in the virtual environment. In fact, the goal is for the computer to redraw the virtual environment you see 30 times a second. Some computers, such as flight simulators, are even faster and can redraw the virtual world 60 times a second. All this work takes a very powerful computer.