The goal of this project is to create practical approaches to human activity recognition (HAR). HAR is the process of allowing IT devices, processes and infrastructures to be aware of the situations and activities of the human beings interacting with these systems. Awareness is typically enabled by applying data processing and machine learning techniques to the output of mobile device sensors being carried by the subject (i.e. a smartphone). This awareness allows systems to better understand the humans interacting with them, thereby improving the response of the system to user input. Ultimately the goal is to have the systems understand the human controllers and their situations so well that explicit input becomes unnecessary.

For practical HAR (PHAR) to become a reality, there are several challenges which need to be addressed. TecO addresses 3 main challenges in this project. The first challenge is to reduce power consumption of the activity recognition process. Battery capacity is quickly becoming the limiting factor for mobile devices as it does not scale with Moore’s Law. In order for such an approach to be practical, it cannot detract from the overall user experience by reducing the battery life of the device running the awareness algorithms. This challenge is specifically applicable to mobile smartphones, although it can be extended all types of mobile devices as well as city-scale applications where power consumption has an impact.

A further challenge is to be able to recognize the activities and situations of groups of users, rather than just individuals. Increasingly instrumented environments, such as meeting rooms, are dealing with multiple users at once. The context of the group can be of vital importance to understanding their requirements on the smart environment, and may also differ greatly from the activities of the individuals in the group. Therefore, activity-aware systems must also take this into consideration and observe groups as well as individuals. There difficulties come with the scale of individuals and devices as groups approach the size of crowds, as well as the complex nature of the interactions between group individuals.

Another challenge addressed by PHAR is that of allowing recognition algorithms to adapt to the humans using them. As time goes on, the behavior of human beings changes, where activities are performed differently, some activities are no longer performed, and some activities. Therefore, the awareness of the devices must be fitted not only to the individual, but must be optimized periodically to make sure they change with the user. PHAR researches using a server-based architecture for activity recognition as a service. A service-based architecture allows individuals to benifit from the progress of the community as a whole by allowing local improvements to be extended to the community. Through modularization, it also allows local intelligence to by extend or cropped to account for appearing and disappearing activities, and for adaptation to changes in existing activities.

The project is funded by, and in collaboration with, SAP Reasearch.

Selected Publications