|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
Group Tele-Immersion
Henry Fuchs
Funding: Department of Energy ASC VIEWS program
Introduction The goal of group tele-immersion (GTI) is to enable high fidelity, immersive teleconferencing between groups at distant locations. Most group teleconferencing systems in use today are simply versions of the one-to-one system used by a group of people at each site. Such single camera/single display systems usually suffer from low resolution, small fields-of-view, and smaller than life-size displays. Attempts to overcome these limitations usually involve replicating the one-to-one system for each set of participants, but this creates hard boundaries between video images that may be disconcerting to viewers. The group tele-immersion project will transmit video data to be reconstructed as a single high-resolution continuous image, for viewing on a life-size projector display for multiple participants between two remote locations.
 Our conceptual vision for group tele-immersion
Initial Design and Implementation
Based on our experience with previous tele-immersion and multi-
projector display wall systems, we developed the following design for a group-
to-group teleconferencing system:
 Initial design concept: 3 projector display with 8 capture cameras mounted above the display
The initial system implementation included:
We present a system and techniques for synthesizing views for many-to-many video teleconferencing. Instead of replicating one-to-one systems for each pair of users, or performing complex 3D scene acquisition, we rely upon user tolerance for soft discontinuities for our rendering techniques. Furthermore, we observed that the participants' eyes usually remain at a constant height (sitting height) during video teleconferencing, thus we only need to be able to synthesize new views along a horizontal plane. To accomplish this, we have developed a real- time system that uses a linear array of cameras to perform Light Field style rendering. Vertical image strips are chosen from each camera and blended to create a continuous single image for display. The simplicity and robustness of Light Field rendering, combined with the natural restrictions of limited view volume in video teleconferencing, allow us to synthesize photo-realistic views for a group of participants at interactive rate.
 Orthogonal view Line Light Field rendering with the lower color band showing the blended contribution from the eleven cameras, each represented by a unique color Acquisition and Rendering Implementation The cameras of the linear array are connected in pairs to commodity PCs, which are interconnected with 100Mbps Ethernet. The cameras are synchronized by a wire controlled from a PC and fully calibrated. The PCs act as video servers and JPEG-encode the video data at full VGA resolution or a smaller region-of-interest specified by the algorithm. The video data is sent to a rendering PC which drives a two projector display using an NVIDIA GeForce4 video card. The rendering algorithm blends these camera images using a Light Field- based technique to create a continuous, high-resolution, wide field-of-view image from a perspective further behind the screen, a good compromise for a group.
 Line Light Field design: Abutted projector display with a dense horizontal camera array at table level and actual camera array implementation
 Remote participants and local participants with display at UNC
Internet Conferencing Architecture
We have implemented a 3D teleconferencing prototype between the
University of Kentucky (UKy) and the UNC-CH. Each site has a total of eight Sony
digital Firewire cameras arranged in a linear array. These cameras are regularly
placed at 65 millimeter apart, very close to the minimum distance allowed by the
form factor of the camera body. All cameras are synchronized by a wire
controlled from a PC and fully calibrated. The video acquisition system (one in
each site) includes four PCs interconnected through 100Mbit Ethernet. Each of
them is connected to two Sony cameras and is used to capture and JPEG-encode
the raw image data at full VGA resolution. The JPEG streams can be sent through
the network.
We use light field rendering to synthesize new images from a user-driven virtual
camera. There are two options to run the view synthesis program. One is to send
all the video streams over the internet and synthesize novel views at the remote
site (remote rendering). Alternatively we can synthesize views locally and only
send the nal result to the remote site (local rendering). The first approach
has a lower latency for changing viewpoint while the second is easier to manage
from a network standpoint. In terms of scalability, the bandwidth requirement
for remote rendering is O(kn) where k is the number of cameras and n is the
number of sites (assuming multicast is used). The bandwidth require- ment for
local rendering, on the other hand, has a quadratic growth rate at O(n2). Given
that our current prototype includes only two sites and the viewpoint is not
changing rapidly (i.e., no head tracking), we chose to implement the local
rendering approach.
Locally, we can achieve an update rate of 4-7 frames per second (fps) for VGA
input images. The bottleneck is in image capture. We can only capture
synchronized VGA resolution images at 7-8 FPS with two cameras on the same 1394
bus. This is caused by the 1394 bus bandwidth limita- tion and the transfer
characteristics of the digital cameras under external trigging. The synthesized
view, typically at 1024x512, is read back from the rendering program's frame-
buffer and sent to the remote site through TCP/IP with JPEG encoding. The frame
rate between UKy and UNC- CH varies from 5 fps to 10 fps, depending on the
network traffic. Optimization in the network code or the use of a more
sophisticated compression scheme is expected to substantially increase the
frame rate.
 Live LLF-based conferencing between UNC-CH and UKy |
|||
|
Updated: 01 May 2005 |
|||