The need to execute Deep Neural Networks (DNNs) at low latency and low power at the edge has spurred the development of new heterogeneous Systems-on-Chips (SoCs) encapsulating a diverse set of hardware accelerators. How to optimally map a DNN onto such multi-accelerator systems is an open problem. We propose ODiMO, a hardware-aware tool that performs a fine-grain mapping across different accelerators on-chip, splitting individual layers and executing them in parallel, to reduce inference energy consumption or latency, while taking into account each accelerator's quantization precision to maintain accuracy. Pareto-optimal networks in the accuracy vs. energy or latency space are pursued for three popular dataset/DNN pairs, and deployed on the DIANA heterogeneous ultra-low power edge AI SoC. We show that ODiMO reduces energy/latency by up to 33%/31% with limited accuracy drop (-0.53%/-0.32%) compared to manual heuristic mappings.
@article{arxiv.2306.05060,
title = {Precision-aware Latency and Energy Balancing on Multi-Accelerator Platforms for DNN Inference},
author = {Matteo Risso and Alessio Burrello and Giuseppe Maria Sarda and Luca Benini and Enrico Macii and Massimo Poncino and Marian Verhelst and Daniele Jahier Pagliari},
journal= {arXiv preprint arXiv:2306.05060},
year = {2023}
}
Comments
Accepted at 2023 ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED)