Registration Errors in Augmented Reality Systems

			    Richard L. Holloway

      Ph.D. dissertation, University of North Carolina at Chapel Hill
	    Under the direction of Dr. Frederick P. Brooks, Jr.

Abstract

Augmented reality (AR) systems combine three-dimensional computer-generated 
imagery with the view of the real environment.  A system designed at the 
University of North Carolina uses a see-through head-mounted display (STHMD) 
to superimpose CT skull data onto the head of the real patient, thereby 
giving the surgeon "x-ray vision", or the ability to "see" the patient's bone 
through the soft tissue.  The promise of such a system is that viewing the 
data in situ will allow surgeons to make better surgical plans because they 
can see the complex relationships between the bone and soft tissue more 
clearly.

A critical problem is that the computer-generated objects do not currently 
remain correctly registered with the real anatomy--objects aligned from one 
viewpoint appear misaligned from another and appear to swim about as the 
viewer moves.  The problem is fundamentally hard:  perfect registration 
requires perfect knowledge of the myriad system parameters and instantaneous, 
perfectly accurate measurements.

This dissertation presents a registration error model for AR systems and uses 
it to gain insight into the nature and severity of the registration error 
caused by the various error sources.  My thesis is that a mathematical error 
model for AR systems enables the system architect to determine
    1.  what the error sources are and which are the most significant, 
    2.  the sensitivity of the net registration error to input errors in each 
    	part of the system,
    3.  the nature of the distortions caused by each type of error, 
    4.  how to best calibrate the system, and
    5.  the level of registration accuracy one can expect as a function of the 
    	input errors.

Application of the model to our surgery planning system yielded the following 
main results:
  - Even for moderate head velocities, delay causes more registration error
    	than all other sources combined;
  - Using the eye's center of rotation as the eyepoint in the computer 
    	graphics model should obviate the need for eye tracking;
  - Tracker error is a significant problem both in head tracking and in 
    	system calibration;
  - The World coordinate system adds error and should be omitted when 
    	possible;
  - Optical distortion is a significant error source, but correcting it in
    	the graphics pipeline often induces delay error larger than the
    	distortion error itself;
  - Knowledge of the nature of the various types of error facilitates 
    	identification and correction of errors in the calibration process.

Although the model was developed for STHMDs for surgical planning, many of
the results are also applicable to other HMD systems.