Optimal deployment of deep neural networks (DNNs) on state-of-the-art Systems-on-Chips (SoCs) is crucial for tiny machine learning (TinyML) at the edge. The complexity of these SoCs makes deployment non-trivial, as they typically contain multiple heterogeneous compute cores with limited, programmer-managed memory to optimize latency and energy efficiency. We propose HTVM - a compiler that merges TVM with DORY to maximize the utilization of heterogeneous accelerators and minimize data movements. HTVM allows deploying the MLPerf(TM) Tiny suite on DIANA, an SoC with a RISC-V CPU, and digital and analog compute-in-memory AI accelerators, at 120x improved performance over plain TVM deployment.
@article{arxiv.2406.07453,
title = {HTVM: Efficient Neural Network Deployment On Heterogeneous TinyML Platforms},
author = {Josse Van Delm and Maarten Vandersteegen and Alessio Burrello and Giuseppe Maria Sarda and Francesco Conti and Daniele Jahier Pagliari and Luca Benini and Marian Verhelst},
journal= {arXiv preprint arXiv:2406.07453},
year = {2024}
}
Comments
Presented at DAC2023. Open-source code is available at https://github.com/KULeuven-MICAS/htvm