New Challenges of Ubiquitous Computing and Augmented Reality
Albrecht Schmidt, Michael Beigl
Telecooperation Office (Teco), University of Karlsruhe, Germany
Introduction
Ubiquitous Computing [W91, W93a, W93b] is an inevitably evolving topic in computing. Looking around in our homes or offices many devices are already equipped with computers. Nevertheless people still use their washing machines and telephones not considering them as a computer. In other parts we don’t agree with Weiser's Vision [W91]. The issue of privacy and security as well as individualism is underestimated and collides with the real world where there is competition and rivalry. As a solution we suggest in this position paper a concept that integrates a personal device (PD) into UbiComp to overcome these problems.
Both terms Ubiquitous Computing (UbiComp) and Augmented Reality (AR) are strongly linked together. We think one important point in UbiComp is that the devices provide smart support to user without forcing the user to change behavior. The access to devices should not be complicated due to the fact that they offer more functionality. Often the opposite is the case - the devices become too complex and people just use the very basic functions [N88], e.g. of modern VCRs. Having this point of view we focus on how computing devices can augment the real reality. Using this approach we emphasize that the way people will interact with their environment is not changed in general; in this way we hope to achieve an intuitive usage paradigm. Our vision of AR takes the term literally - this is a major contrast to most other researchers working in AR. They focus mainly on how virtual features are useful to enhance the reality, for more information see [F93, F93b, S91] or the AR-Introduction [V98].
For us the following four points are central to our vision of UbiComp and AR.
Looking at the first three topics we consider the usage of an accompanying personal device (PD) as a key issue. Considering the last point the introduction of an additional PD seems contradictory. But only at the first sight! The PD can ease the interaction and the usage of ordinary devices. For applications where there is need for a more sophisticated additional user interface, the PD is an appropriate solution. That means the user can have a device according to his needs and preferences. He only needs to know how to handle one single device that is customized for him.
Environment and Context Aware Computing
The environment and the context surrounding the user can give cues to enhance usability. The devices can sense what the user does need next, what she is probably going to do, and what actions can be taken to ease the task of the user [M97a, M97b].
Systems that sense the whereabouts of the user (the actual location) are already on the market (GPS, active badges). Using this information and knowing the goal of the user (the destination) the system can give advice where to proceed to find the way. Examples for implementations of such systems are GPS controlled car navigation system [B98, N98], the NaviCam [B97, R95], or location aware computing [B97b].
Looking into future applications the statement Context is more than position [A97] becomes central. We consider as context: the people around the user, the situation (e.g. in a meeting, making a phone call, or having a coffee, walking), the environment (e.g. location, temperature, time, or light), how the user feels (e.g. pulse, body temperature, or skin resistance). This additional information about the context can make the computing device act and react more sophisticated and less obtrusive.
The precious information given by the context must be acquired using the PD. We propose that the PD has a sensor system to sense the environment. Within the European TEA-Project [EP26900] we are researching location and situation awareness for different mobiles devices (Personal Digital Assistance, Mobile phones, Handheld Computers). The approach goes beyond the location; the information given by other features of the environment such as the people surrounding the user or the temperature gives a context to all activities and documents.
There are numerous examples for such applications. Retrieving information can be enhanced; using these features a document may be queried describing the context in that it was created additional to the content. E.g. "I am looking for some notes, I just remember it was freezing cold and the two guys from Nokia telecom were with me". Furthermore having knowledge of the environment the PD can adjust also the UI to the users needs. If the PD recognizes that the user is driving a car the system can switch to voice communication or if it is recognized that the user is in a meeting the phone will not ring. Other applications for selective communication or context-sensitive to-do list are easily to imagine.
A major concern of people confronted with the ideas of UbiComp is their privacy. The thought that all steps and actions of somebody are monitored and stored for further use is frightening - for an idea just see what is possible with active badges [O98]. In fact using ideas of UbiComp can provide the means that result in a total monitoring of a person.
To overcome these concerns the potential users have to trust the system. For this we see two important factors.
First it is necessary to provide a privacy and security model that is understandable by the user. It is of major importance that the user can comprehend that he has the control over all information concerning him. And that it is his choice what information he wants others to know. Second the privacy must not be some add-on; it must be modeled into the system at architectural level. We consider a PD as a step towards a solution. The user can take (physically!) his data with him. Combining this with new usage paradigms such as pick-and-drop [R97, R98] the user can easily decide which information to share and which to keep secret.
It is also necessary that the user understand the trade-off between automatic processes and privacy, as depicted in Figure 1. An example of such a trade-off is the usage of an electronic diary. If a user allows everybody full access to his diary meeting scheduling can be automated easily, but on the other hand everybody will know all about his life.
Figure 1: Automation and Privacy
Augmented Human-to-Human Interaction.
Additional the ordinary interfaces the accompanying personal device may provide further functionality to the user, such as wallet, identity card, briefcase or notepad. This device is not a substitute for the real devices it is just an additional device that offers these functions.
PDs have to have communication facilities to communicate with other users easily. This functionality is implemented in a very rudimentary way with infrared interfaces (IrDA) or even less convenient sync cradles that are connected to a fixed computer. For a good support of the user the communication facilities of the PD in a heterogeneous environment must be improved significantly. There is need for communication between different PDs, e.g. for exchanging documents between different users or giving money to somebody. In this area protocols have to be developed that support heterogeneous communication. In a case study we implemented data-exchange tool to support communication between a HP620LX running WindowsCE and a Psion Handheld computer [L98]. Using PDs as electronic wallets and identity cards the communication needs also authentication and encryption mechanisms.
New usage paradigms such as Pick-and-Drop [R97] will ease the data-transfer between different users on user interface level.
Heterogeneous and Distributed User Interfaces
In UbiComp scenarios users interact with many different devices to fulfil a task. The user could use all surrounding devices and objects as an interface. For example, a user entering a room through a door uses the door handle as an interface. When using the door handle he interacts with the system, giving some information that can be used by an application. This is comparable to moving a mouse pointer in common UI.
Therefore real thinks in the environment like the door in a building, the light switch, or the window become part of the distributed UI. As it can be seen from the example above UbiComp applications have to interact with many distributed and heterogeneous user interface devices. One important problem here is that there are neither available user interface design methods nor tools that could assist a design of these kinds of applications. We have presented a first system for solving these problems in [B98b]. For collecting the information given by the user to any UI component (like opening a door or switching on the light) a low-bandwidth data transfer from the distributed UI to the PD is desirable.
Focusing on the issue from a more user centric view it is common sense that users like to interact with applications with the user interface they are used to. In our opinion UbiComp applications should therefore be controlled by the user manipulating the controls he is used to. The introduction of new user interfaces - if necessary - should be additional to ordinary already used interface; the user should be given the choice, which interface to use.
A scenario often quoted in ubiquitous computing is the control of the house environment. One application in this scenario is the regulation of the temperature in a house. Today a user operates the switch control on the heating to heat a room and opens the window to cool a room - for these UI we use the term ordinary UI. No user needs a training to operate this UI. These controls are in every house and in the UbiComp approach they are just connected to a computer system or are equiped with a computer. Using the information collected from the distributed UI an intelligent system can control the temperature on behave of the user. The wishes of the user are derived from the former behaviour of the user. If the user feels uncomfortable with the temperature the system has adjusted, the user interacts with the ordinary interfaces using heating control or the window handle. Interacting with the system only when the system behaves not like wanted is an inversion of the prior principle to take control of all house functionality actively. Despite of that the effect to the user is the same: The temperature in the room is comfortable for him and he could adjust the temperature with the controls he is used to. Additional to the ordinary interface the user could be provided with a more sophisticated UI - integrated in his personal device. This offers the user the possibility to change his behaviour, but does not force him to do so. Using information gathered by other UI-components (e.g. the door handle) the system could provide even more sophisticated control. E.g. the system could save energy by lowering the temperature of empty rooms.
Taking into account the ideas noted above, new UbiComp applications could be created – and even without annoying the user with new and more complex interfaces.
Conclusion
We propose that an accompanying personal device provides important advantages in UbiComp scenarios. PD can support the usage of context and environment information for new applications. To model an easy-to-understand security and privacy concept we consider a personal device as essential. The aspect of augmented human-to-human interactions creates new challenges for heterogeneous communication of mobile devices. We think the user should not be forced to change the behavior, therefore we consider the integration of ordinary UIs aside with more sophisticated new user interfaces as central.
References
[A97] G. D. Abowd. Ubiquitous Computing: Research Themes and Open Issues From an Application Perspective. ftp://gvu.gatech.edu/pub/tr/96-24.ps.Z
[B97] S. V. Brull. SONY: A LITTLE SENSOR WITH A BIG FUTURE. Businessweek 06/23/97. http://www.businessweek.com/1997/25/b353217.htm 1997.
[B97b] H.W.P. Beadle, G.Q. Maguire Jr., M.T. Smith. Using Location and Environment Awareness in Mobile Communications, Proc. EEE/IEEE ICICS'97, Singapore, September 1997, pp. 1781-1785. http://www.elec.uow.edu.au/people/staff/beadle/badge/icics97/badge/
[B98] BMW. The BMW navigation system. BMW compass. http://www.bmw.com/compass/htdocs/BMWe/backissue/FORSCH2E.shtml, 1998.
[B98b] M. Beigl, Ein Ubicom Interface Design Vorschlag. Klausurtagung des Instituts für Telematik. 29.-31.3.1998. Mainz. Universität Karlsruhe. 1998.
[EP26900] Esprit Project 26900: Technology for Enabling Awareness (TEA).
[F93] S. Feiner, B. MacIntyre, et al. (1993). Windows on the World: 2D Windows for 3D Augmented Reality. Proceedings of ACM Symposium on User Interface Software and Technology. Atlanta, GA, Association for Computing Machinery: 145-155.
[F93b] S. Feiner, B. MacIntyre, D. Seligmann. Knowledge-Based Augmented Reality. CACM, 36(7):52-63. 1993.
[L98] M Lauff, L. Johannsen, K. Müller: IrDA Kommunikation in heterogenen Umgebungen am Beispiel von Windows CE und EPOC32, Internal report at TecO, 1998
[M97a] P. Maes. MIT Media Lab, Software Agents Group: Research (1997) http://agents.www.media.mit.edu/groups/agents/
[M97b] P. Maes on Software Agents: Humanizing the Global Computer. IEEE Internet Computing July-August (1997b)
[N88] D. A. Norman. The Psychology of Everyday Things. Basic Books. New York, 1988.
[N98] K. Nagao and J. Rekimoto. Agent Augmented Reality: A Software Agent Meets the Real World. Sony Computer Science Laboratory Inc. http://www.csl.sony.co.jp/person/nagao/icmas96/outline.html, 1998.cdexcd
[O98] Olivetti. A stuff location system at ORL. http://www.orl.co.uk/cgi-bin/ab. http://www.orl.co.uk/ab.html 1998.
[R95] J. Rekimoto, "Augmented Interaction: Interacting with the real world through a computer", HCI International, 1995.
[R97] J. Rekimoto, "Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments", Proceedings of UIST'97, pp. 31-39, 1997.
[R98] J. Rekimoto. "A Multiple Device Approach for Supporting Whiteboard-based Interactions", CHI'98, 1998
[S91] Seligmann, D. and Feiner, S. Automated generation of intent-based 3D illustrations. Computer Graphics, 25(4), July 1991 (Proc. ACM SIGGRAPH '91, Las Vegas, NV, July 28-August 2, 1991), 123-132.
[V98] J. Vallino. Introduction to Augmented
Reality. 1998
http://www.cs.rochester.edu/u/www/u/vallino/research/AR/introduction.html
[W91] Mark Weiser, "The Computer for the Twenty-First Century," Scientific American, pp. 94-10, September 1991.
[W93a] Mark Weiser, "Some Computer Science Problems in Ubiquitous Computing," Communications of the ACM, July 1993.
[W93b] Mark Weiser, "Hot Topics: Ubiquitous Computing" IEEE Computer, October 1993.
Albrecht Schmidt, Michael Beigl: New Challenges of Ubiquitous Computing and Augmented Reality. 5th CaberNet Radicals Workshop, 5-8 July 1998, Valadares, NR. Porto, Portugal