Background

Human Activity Recognition (HAR) uses motion sensors, such as a smartwatch’s accelerometer, to recognize activities like walking or cooking, often with neural networks running directly on the device. The standard approach cuts the sensor signal into short, overlapping windows and classifies each window on its own. Recent work has questioned this by evaluating models in the real inference setting of complete recordings, showing that context from previous windows makes predictions more accurate and more consistent [1]. This raises the question of why each window should be processed on its own at all.

This thesis explores running HAR neural networks directly on the continuous sensor stream. Because the windows overlap, the windowed approach processes every sensor sample several times, while a streaming network processes each sample only once and saves computation. This works for purely convolutional networks, which are among the most accurate and efficient HAR models [2, 3, 4], since their filters compute the same values on the overlapping parts of windows [5]. Streaming keyword spotting has already shown this to work in audio [7]. The streaming setup also makes it easy to add memory across windows on top, using a small recurrent neural network (GRU [6]) for smoother predictions. The approach will be applied to existing networks such as CNN-HAR [3] and LiteWay [4] and evaluated on several HAR datasets for accuracy on complete recordings [1] and computational cost on microcontrollers (e.g., with TensorFlow Lite Micro). The goal is a simple, general method that makes HAR neural networks cheaper to run on wearables while giving smoother, more reliable predictions.

References
[1] Bock, M., Gall, J., Moeller, M., & Van Laerhoven, K. “DeepConvContext: A Multi-Scale Approach to Timeseries Classification in Human Activity Recognition.” ISWC, 2026.
[2] HARena: A Living Benchmark for Sensor-Based Human Activity Recognition on Resource-Constrained Edge Devices. Under review.
[3] Yang, J. B., et al. “Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition.” IJCAI, 2015.
[4] Nshimyimana, D., Fortes Rey, V., Liu, M., Zhou, B., & Lukowicz, P. “LiteWay: Lightweight HAR via Temporal Efficient highWAY.” ISWC, 2026.
[5] Long, J., Shelhamer, E., & Darrell, T. “Fully Convolutional Networks for Semantic Segmentation.” CVPR, 2015.
[6] Cho, K., et al. “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation.” EMNLP, 2014.
[7] Rybakov, O., Kononenko, N., Subrahmanya, N., Visontai, M., & Laurenzo, S. “Streaming Keyword Spotting on Mobile Devices.” Interspeech, 2020.

Tasks

  • Review HAR models and streaming inference (e.g., keyword spotting), and work out the conditions for a CNN to give identical results on a stream.
  • Build streaming versions of existing HAR networks (e.g., CNN-HAR, LiteWay) and verify them against window-based outputs.
  • Evaluate accuracy and computational cost against sliding windows on several HAR datasets.
  • Deploy on a microcontroller with TensorFlow Lite Micro and measure latency and memory.
  • Add memory across windows as in DeepConvContext (e.g., GRU), analyze how it works with streaming, and evaluate on complete recordings.

Requirements

  • Basic knowledge of machine learning and neural networks.
  • Good Python skills, basic C/C++ knowledge; experience with PyTorch or TensorFlow is a plus.
  • Interest in sensor data, wearables, microcontrollers and efficient neural networks.

Application

Please include a short paragraph explaining your motivation, your CV, your study program (Bachelor/Master), current semester and field of study, a transcript of records with courses and grades, your programming experience, and any areas of interest relevant to the topic.

Contact