Display registration for device interaction a proof of principle prototype

A method is proposed to facilitate visually-driven interactions between two devices, which we call the client, such as a mobile phone or personal digital assistant (PDA), which must be equipped with a camera, and the server, such as a personal computer (PC) or intelligent display. The technique that we describe here requires a camera on the client to view the display on the server, such that either the client or the server (or both) can compute exactly which part of the server display is being viewed. The server display and the clients image of the server display, which can be written onto (part of) the client's display are then registered. This basic principle, which we call "display registration" supports a very broad range of interactions (depending on the context in which the system is operating) and it will make these interactions significantly quicker, easier and more intuitive for the user to initiate and control. In addition, either the client or the server (or both) can compute the six degree-of-freedom (6 DOF) position of the client camera with respect to the server display. We have built a prototype which proves the principle and usefulness of display registration. This system employs markers on the server display for fast registration and it has been used to demonstrate a variety of operations, such as selecting and zooming into images.