Deep Learning Training on the Edge with Low-Precision Posits

Hamed F. Langroudi; Zachariah Carmichael; Dhireesha Kudithipudi

Deep Learning Training on the Edge with Low-Precision Posits

Machine Learning 2019-08-01 v1 Machine Learning

Authors: Hamed F. Langroudi , Zachariah Carmichael , Dhireesha Kudithipudi

Abstract

Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and Fashion MNIST corpuses, where 16-bit posits outperform 16-bit floating point for end-to-end DNN training.

Keywords

deep learning mixed precision training

Cite

@article{arxiv.1907.13216,
  title  = {Deep Learning Training on the Edge with Low-Precision Posits},
  author = {Hamed F. Langroudi and Zachariah Carmichael and Dhireesha Kudithipudi},
  journal= {arXiv preprint arXiv:1907.13216},
  year   = {2019}
}

Related papers

View all related →