English

Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition

Machine Learning 2025-05-30 v1 Artificial Intelligence Performance

Abstract

This paper aims to develop an energy-efficient classifier for time-series data by introducing PatchEchoClassifier, a novel model that leverages a reservoir-based mechanism known as the Echo State Network (ESN). The model is designed for human activity recognition (HAR) using one-dimensional sensor signals and incorporates a tokenizer to extract patch-level representations. To train the model efficiently, we propose a knowledge distillation framework that transfers knowledge from a high-capacity MLP-Mixer teacher to the lightweight reservoir-based student model. Experimental evaluations on multiple HAR datasets demonstrate that our model achieves over 80 percent accuracy while significantly reducing computational cost. Notably, PatchEchoClassifier requires only about one-sixth of the floating point operations (FLOPS) compared to DeepConvLSTM, a widely used convolutional baseline. These results suggest that PatchEchoClassifier is a promising solution for real-time and energy-efficient human activity recognition in edge computing environments.

Keywords

Cite

@article{arxiv.2505.22985,
  title  = {Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition},
  author = {Masaharu Kagiyama and Tsuyoshi Okita},
  journal= {arXiv preprint arXiv:2505.22985},
  year   = {2025}
}

Comments

23 pages,5 figures

R2 v1 2026-07-01T02:47:36.232Z