User-interface Generation for Mobile and Desktop Computing

Principal Investigator: Prasun Dewan
Funding Agency: National Science Foundation
Agency Number: IIS-0312328

Abstract
This project addresses research in user-interface abstractions, which play a fundamental role shaping the nature of the user-interfaces we use and the programs we write. Motivated by the needs of smart spaces consisting of networked appliances and interactive mobile devices, we are proposing to revisit the idea of user-interface generation. Instead of suggesting a single, fixed user-interface generator for all applications, we are proposing multiple domain-specific generators. We are developing a general user-interface architecture that allows different generators to share code and their clients to incrementally customize them. To ensure the generality of the architecture, we are considering the needs of both mobile and desktop computing. In particular, we are considering two domains of desktop computing for which user-interface generation seems to be particularly well suited - research applications and teaching of introductory programming. The architecture is based a model of user- interaction that treats user-interfaces as editors of logical structures of objects. It consists of several layers including external encapsulated application objects, a set of logical structures, editors of these structures, and an external toolkit. In addition, it includes objects that automatically connect objects in adjacent layers. Our hypothesis is that emergence of programming and user-interface conventions allows these connections to be made more or less automatically for the vast majority of applications in the domains we are considering. We are developing an algorithm for automatically identifying the behavior of the external application and toolkit objects from the patterns used in the signatures of the methods of these objects. Unlike the traditional approach of requiring an application to adapt to the (programming) interfaces provided by the tool, this algorithm allows the tool to adapt to the interfaces provided by the application. We are also providing an axiomatic scheme for formally specifying the default patterns assumed by the generator, which can be customized to specify domain-specific patterns and associated semantics. Based on these formal specifications, we are developing parameterized algorithms for creating textual and graphical presentations of application objects, and automatically implementing commands to edit these presentations, including commands to undo and synchronize these objects, which require an understanding of the semantics of objects. We are developing efficient implementation techniques to ensure that the cost of generating and using a user-interface on a mobile device is low. In particular, we are developing a technique to retarget a user-interface generated for a particular appliance to a new appliance with the same logical. structure. We will evaluate our work by identifying a representative set of applications and devices from each of the domains we are addressing. In particular, we will generate the user-interface of the next version of the nanoManipulator collaboratory, a research application being built at UNC. We will determine how well our work supports automation, incremental customization, flexibility, performance, and integration with existing programming abtractions.