The flow of ideas is established between paragraphs.

The multi-sensor tracker is a tracking system that captures users’ motion, for example, the users’ head poses (position and orientation) are tracked as they move around a room. The tracker provides the user’s pose to applications, such as virtual reality (VR) systems that generate environments in which the user is visually immersed. The VR application needs accurate and frequent pose measurements to generate several images per second to display to the user. 

Several design challenges need to be addressed in order to have a successful tracker that can frequently and accurately report the user’s pose. The tracker must minimize the delay from when the tracker’s sensors acquire input, which represents the user’s current pose, to when the user’s pose is computed. Every time the user moves, the tracker’s input is invalidated. The tracker’s input is from sensors that acquire data of reference points worn by the user. The sensors should acquire minimal data in order to reduce the computation for each pose measurement. The information the sensors gather is distorted by disturbances that can occur in the tracking volume. The quality of the signal from the tracked reference points is also degraded when the volume in which the user is tracked increases and therefore the distance between the sensor and tracked point increases. Although the tracked reference points on the user could be engineered to give the sensor a strong signal reading, the engineered gadget becomes bulky and unnatural for the user to wear.

The envisioned multi-sensor tracker will be scalable able to track several users in a large volume. In addition, the tracker will be robust to disturbances that occur in the tracking environment. The multi-sensor tracker consists of a network of several small simple sensors that track reference markers on the user. The sensors are distributed throughout the tracking volume to ensure a clear sensor signal by minimizing the distance between the tracked points on the user and the sensor. The reference points become simpler and therefore become more comfortable to wear. Each sensor will be attached to a processing unit. Distributing the tracking computation among the sensors’ processors minimizes the computation delay of the user’s pose. The partial tracking information computed on each processing unit is communicated through the network connecting the sensors. Some sensors will combine the partial results into the complete result representing the tracked user’s pose. The latency to compute the tracked pose should not increase as more sensors are added to the tracker. Hence, the system will scale to track in a large volume. 

The multi-sensor tracker will use the SCATT (Single constraint at a time) theory as a statistical estimation model to compute the user’s pose form the sensor readings. SCAAT was developed at UNC and is successfully applied in the HiBall tracker’s tracking algorithm. The benefit of SCAAT over other tracking algorithms is low latency to compute the tracked pose and an increased sampling rate of the tracker. SCAAT’s contribution is to compute the user’s pose after every sensor reading, although each sensor only has partial information of the user’s complete pose. 

In the multi-sensor tracker the SCAAT model will execute on the individual processing units of the sensors. After each sensor reading, the tracker will be able to output an update of the tracked user’s pose. The theory of SCAAT, however, will need to be extended to enable distributed processing as opposed to the current centralized processing model it uses. 

The distributed version of SCAAT requires a network connecting the sensors to enable them to communicate their data to other sensors that will complete the tracking computation. Concepts from directed diffusion are suitable to design the network connecting the sensors. Directed diffusion networks were designed to support systems, like the multi-senor tracker, consisting of several sensors accomplishing a common task. The directed diffusion network has to be adapted to suit the particular needs of the multi-sensor tracker. 

The effectiveness of applying SCAAT and directed diffusion concepts to the multi-sensor tracker will be evaluated by its scalability, tracking stability, the human effort to initialize the system. The scalability of the tracker is evaluated by how many sensors can be added to the system. More sensors enable tracking the user’s pose in a larger volume and increase the tracking accuracy. Each sensor will require bandwidth from the network, and sensors can be added until the network bandwidth limit is reached. A tracking system of many sensors is more appealing if initializing the state of the individual sensors can be automated. A stable tracker needs to continue to accurately track even when there are obstructions to the sensors.

Before designing the multi-sensor tracker with the SCAAT and directed diffusion concepts, it is necessary to know the tracker’s performance specification. This paper uses the Perlin auto stereoscopic display to demonstrate the required accuracy of the user’s pose and sampling frequency of the multi-sensor tracker. The auto stereoscopic display enables a user to see a 3D image on a computer screen. The display application needs the user’s pose to generate the 3D image.

This paper explores the design issues of the multi-sensor tracker, currently only a concept, in the following five sections. The background material in section 2 compares the multi-sensor tracker to other trackers and explains the details of SCAAT and directed diffusion. Section 3 specifies the tracker’s required performance according to the auto stereoscopic display. A description of how the multi-sensor tracker works is in section 4. Section 5 discusses the design choices of the directed diffusion network to provide the appropriate communication for the SCAAT tracking algorithm. The conclusion and future work are in the last section.