This work targets the automated minimum-energy optimization of Quantized Neural Networks (QNNs) - networks using low precision weights and activations. These networks are trained from scratch at an arbitrary fixed point precision. At iso-accuracy, QNNs using fewer bits require deeper and wider network architectures than networks using higher precision operators, while they require less complex arithmetic and less bits per weights. This fundamental trade-off is analyzed and quantified to find the minimum energy QNN for any benchmark and hence optimize energy-efficiency. To this end, the energy consumption of inference is modeled for a generic hardware platform. This allows drawing several conclusions across different benchmarks. First, energy consumption varies orders of magnitude at iso-accuracy depending on the number of bits used in the QNN. Second, in a typical system, BinaryNets or int4 implementations lead to the minimum energy solution, outperforming int8 networks up to 2-10x at iso-accuracy. All code used for QNN training is available from https://github.com/BertMoons.
@article{arxiv.1711.00215,
title = {Minimum Energy Quantized Neural Networks},
author = {Bert Moons and Koen Goetschalckx and Nick Van Berckelaer and Marian Verhelst},
journal= {arXiv preprint arXiv:1711.00215},
year = {2017}
}
Comments
preprint for work presented at the 51st Asilomar Conference on Signals, Systems and Computers