Personal Information Environments

The pattern of how users interact with computers has changed over time. Users initially shared computers, but increasing capabilities and decreasing prices led to computers dedicated to a single user and the model of the "personal computer". As capabilities increase and prices decrease further, users are now starting to interact with a heterogeneous collection of computing devices (e.g. desktops, laptops, tablets, PDAs, mobile phones).

Despite this latest change, the models of the shared and the personal computer still predominate today. A user purchasing a new computer can configure it with relative ease to serve as either a shared (e.g. among family members) or personal computer. However, current computing devices provide users with little or no support for interacting across a collection of multiple personal devices. The user of that new computer, for example, has no mechanism to inform it of the identities of the other computing devices he employs.

We believe that computing devices need a new model that recognizes that users may employ multiple devices. We propose the model of a personal information environment (PIE) composed of multiple heterogeneous computing devices employed by the same user. We focus on employed, rather than owned, devices because users may regularly employ devices (e.g. work computers) owned by others. A PIE should allow users to easily employ the best device or combination of devices to accomplish tasks. Toward that end, the devices in a PIE should be aware of the existence and availability of each other and provide services that support interacting across them..

While some existing applications support aspects of a PIE, they are individual points in the design space. We currently lack a general infrastructure that focuses on supporting interaction across multiple personal devices. Exploring the design space for PIEs is difficult without such an infrastructure, because researchers and developers must develop capabilities for managing and communicating across personal devices for each new application.

We want to facilitate that exploration in order to both support and leverage the trend of interacting with multiple personal devices. Toward that end, we are developing a prototype Personal Information Environments infrastructure that consists of three primary components: a modified instant messaging server, a client that runs on each device and acts as a conduit for services on that device to communicate, and services that developers create to provide desired multi-device functionality. We have also developed an initial set of multi-device services to demonstrate the capabilities of our infrastructure.

• Jeff Pierce: project lead, infrastructure design and development, example services (jspierce at us.ibm.com)
• Jeff Nichols: example services

Our prototype infrastructure contains three key architectural elements:

1. A PIE server responsible for associating devices with users and routing messages between devices.
2. PIE clients that run locally on (or represent) each device and act as conduits for services to communicate.
3. PIE services that developers create to provide multi-device functionality.

PIE Server

The PIE server is a modified instant messaging server. It provides traditional IM functionality such as registering and authenticating users, maintaining and providing presence data, and routing messages between entities. To that functionality our infrastructure adds support for persistently affiliating devices with users, the ability to define aliases that simplify message addressing, and improved support for asynchronous communication (including across the PIEs of multiple users).

PIE Client

Each of a user’s devices runs (or is represented by) a PIE client. The client provides a user interface that allows users to view the status of their devices, add or remove devices, and manage available services. Toward that end, the client maintains a local copy of the user's device roster: the identity and availability of devices in the user's PIE. It also maintains a list of available services on that device that users can configure, start, and stop.

The client also acts as a conduit for services to exchange messages. To send a message, a service sends the recipient address(es) and message content to the client. The client then handles formatting the actual message and transmitting it to the server (the clients and server use XMPP to communicate). When the client receives a message from the server, it first checks the type of message. If the message contains presence data about one of the user's devices, the client updates its local copy of the device roster and forwards the message to each of the currently active services. For other message types, the client checks the XML namespace(s) in the message and sends copies of it to active local services that expressed interest in any of those namespaces.

We currently have PIE clients than integrate into Sametime 7.5.1 and Notes 8 on Windows, OS X, and Linux devices, and we have a standalone client for Windows Mobile devices.

PIE Services

The PIE client provides support for two types of services: dependent and independent (shown as green and yellow respectively in the architecture image). The client automatically instantiates and initializes dependent services when it starts. Because the client instantiates these services itself, they run in the same process and can access each other's APIs directly through handles on the relevant instances. However, dependent services cannot run independently and must be implemented such that the client can directly instantiate them.

To date we have implemented nineteen PIE services ranging in complexity from tens to thousands of lines of code. Those services include:

• A service that allows users to cut-and-paste or drag-and-drop text, URLs, and files between their devices.

  

  
  
• A service that allows users to remotely browse their devices and retrieve files of interest.

  

  
  
• A Notebooks service that allows users to create and access multiple text notebooks across any of their devices. The service shares changes to a notebook on any particular device with the user's other devices.

  

  
  

• A Shared Lists service that allows users to create and access lists of text items across any of their devices. The service broadcasts changes to lists or to list items (such as checking off an item) to the user's other devices. In addition, the service allows users to share individual lists with other users. For example, a husband and wife could share a grocery list. The husband could add items to the list during the day, and when the wife stops by the store on her way home from work she could use her mobile phone to access the most recent version of the list.

  

  
  

• A Send a URL service that allows users to tell any of their currently available devices to open a web browser and access a URL. Users can specify the desired URL and device using either a standalone GUI or a Firefox plug-in.

  

  
  

• A File Synchronization service that allows users to synchronize different sets of files across their personal devices. Users can choose which of their devices participate for each set of files.

  

  
  

• A mechanism that allows users to search for and retrieve files from their other devices. Users initiate a search by typing keywords into a Search Console service on one device; that service sends those keywords to a Search service on the user's other available devices with instructions to return likely matches. Depending on a receiving device's operating system, a particular Search service searches locally for the keywords using Google Desktop Search, OS X's Spotlight, or Beagle and returns descriptions of candidate matches to the originating device. The Search Console service aggregates and presents the results to the user. The user can choose a particular result and tell the service to transfer that file and open it (or open the specified URL, for search applications that index visited URLs).

  

  
  

• A Recent Shortcuts service that shares across the user's devices the contact information of people that the user has recently emailed or instant messaged, the identity of attachments in email messages that the user has recently viewed, and the identity of files that the user has recently accessed. This service extends the single-device Recent Shortcuts application designed by John Tang, James Lin, Steve Whittaker and Clemens Drew across multiple, heterogeneous devices. The service synthesizes the information from all of the user's devices and presents the results in a single GUI. The user can click on an individual's contact information to send an email or IM to that person or double-click on a listed attachment or recent file to retrieve and open that file.

  

  
  

1. John C. Tang, James Lin, Jeffrey S. Pierce, Steve Whittaker, and Clemens Drews. Recent Shortcuts: Using Recent Interactions to Support Shared Activities. CHI 2007, pages 1264-1272.
2. David Dearman and Jeffrey S. Pierce. "It's on my other computer!": Computing with Multiple Devices. CHI 2008, pages 1144-1153.
3. Jeffrey S. Pierce and Jeffrey Nichols. An Infrastructure for Extending Applications' User Experiences Across Multiple Personal Devices. UIST 2008, pages 101-110.