One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate Attention-based models in a tinyML power envelope with an octa-core cluster coupled with an accelerator for quantized Attention. Our deployment flow enables end-to-end 8-bit Transformer inference, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154 GOp/s (0.65 V, 22 nm FD-SOI technology).
@article{arxiv.2408.02473,
title = {Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow},
author = {Philip Wiese and Gamze İslamoğlu and Moritz Scherer and Luka Macan and Victor J. B. Jung and Alessio Burrello and Francesco Conti and Luca Benini},
journal= {arXiv preprint arXiv:2408.02473},
year = {2025}
}
Comments
Accepted for publication in the SI: tinyML (S1) issue of IEEE Design & Test