基于IntelCaffe的卷积神经网络高效8位低精度推理
计算机视觉与模式识别
2018-05-23 v1
摘要
深度神经网络的高吞吐量和低延迟推理对于深度学习应用的部署至关重要。本文介绍了IntelCaffe的高效推理技术，这是首个支持在Intel Xeon可扩展处理器上高效8位低精度推理及卷积神经网络模型优化技术的Intel优化深度学习框架。8位优化模型通过校准过程从FP32模型自动生成，无需微调或重训练。我们表明，相较于IntelCaffe FP32基线，ResNet-50、Inception-v3和SSD的推理吞吐量和延迟分别提升了1.38倍-2.9倍和1.35倍-3倍；相较于BVLC Caffe分别提升了56倍-75倍和26倍-37倍，且精度损失可忽略。所有这些技术已在IntelCaffe GitHub1上开源，并提供了工件以在Amazon AWS Cloud上复现结果。
引用
@article{arxiv.1805.08691,
title = {Highly Efficient 8-bit Low Precision Inference of Convolutional Neural Networks with IntelCaffe},
author = {Jiong Gong and Haihao Shen and Guoming Zhang and Xiaoli Liu and Shane Li and Ge Jin and Niharika Maheshwari and Evarist Fomenko and Eden Segal},
journal= {arXiv preprint arXiv:1805.08691},
year = {2018}
}
备注
1st Reproducible Tournament on Pareto-efficient Image Classification, co-held with ASPLOS 2018