Evaluating the Performance of Distributed Synchronous Collaboration Architecture

Principal Investigator: Prasun Dewan
Funding Agency: National Science Foundation
Agency Number: IIS-0712794

Abstract
While there has recently been a proliferation of research and industrial-strength collaborative systems, there has been little work done in comparing the performance of their distributed architectures, even though lack of adequate performance can be a show-stopper in many situations. In this project, we will evaluate a design space of architectures that includes existing and potentially useful future architectures. This space will be an extension of the existing zipper model, which defines two dimensions: the level of the logically shared layer of abstraction, and whether this layer is physically centralized or replicated. Intellectual merit: The zipper extensions we will study will include both architectural aspects found today in industrialstrength systems such as push and pull-based multicasting; and several new aspects allowing (a) a replica to be shared by a cluster of users, (b) different pairs of user sites to independently choose shared layers, (c) a layer to serve multiple higher-level layers that can be replicated independently, (d) sharing to be done simultaneously in multiple layers, and (e) a replica to serve a remote but not local user. We will use several metrics to judge the performance of a collaborative application including join, leave, response, feedthrough, and task-completion times. We will develop equations that describe these times as functions of various collaboration parameters. These parameters will include system parameters such as processing powers and network delays; and task parameters such as think times, input processing costs, and number of active and passive users. These equations will be simplified for frequent collaboration conditions such as large, constant, and insignificant think times, network delays, and processing costs. We will perform experiments to validate these equations using logs of different kinds of collaborations such as single-user presentation, multi-user chat, and collaborative visualization. The experimental test-bed will be our current log-based infrastructure modified to support zipper extensions we study. By creating a single system that supports the extended zipper space, we will be able to make experimental performance comparisons that were not possible before. The four components of our work – the space of studied architectures, the analytical model, the experiments, and the experimental testbed will have a symbiotic relationship among them. The analytical model defined for an initial version of the space will identify new useful points in it. Experiments will be used to identify (a) collaboration parameters that perceptibly influence performance and (b) validate the model under various collaborations scenarios. The analytical model will, in turn, be used to speed up experiments and obtain scalability results. Broader Impact: Our research will allow application and infrastructure developers to determine the range of architecture they should support. It will provide scientists, engineers, writers, decision makers and other potential users of collaboration technology a better understanding of the (a) similarities and differences between the architectures of different collaborative systems so that they can choose the correct architecture based on their needs, and (b) consequences of choosing a particular architecture so that there are fewer surprises when problems are encountered. Predictions will be based on previous logs of interaction and general conditions of the collaboration such as single-user presentation vs. multi-user decision making. In addition, our infrastructure will form (a) the first proof-of-concept system supporting the entire space of zipper and non-zipper architectures we will study, (b) a research test-bed for experimentation and (c) a vehicle for teaching collaboration architectures. We have a track record of creating useful, usable, and well-published software that has been used for teaching and research. Finally, we will create a significant number of logs of multi-user activities, which will be publicized so that they can become benchmarks used to compare performance of new architectures.