English

Quantaized Winograd/Toom-Cook Convolution for DNNs: Beyond Canonical Polynomials Base

Machine Learning 2020-04-24 v1 Numerical Analysis Numerical Analysis Machine Learning

Abstract

The problem how to speed up the convolution computations in Deep Neural Networks is widely investigated in recent years. The Winograd convolution algorithm is a common used method that significantly reduces time consumption. However, it suffers from a problem with numerical accuracy particularly for lower precisions. In this paper we present the application of base change technique for quantized Winograd-aware training model. We show that we can train the 88 bit quantized network to nearly the same accuracy (up to 0.5% loss) for tested network (Resnet18) and dataset (CIFAR10) as for quantized direct convolution with few additional operations in pre/post transformations. Keeping Hadamard product on 99 bits allow us to obtain the same accuracy as for direct convolution.

Keywords

Cite

@article{arxiv.2004.11077,
  title  = {Quantaized Winograd/Toom-Cook Convolution for DNNs: Beyond Canonical Polynomials Base},
  author = {Barbara Barabasz},
  journal= {arXiv preprint arXiv:2004.11077},
  year   = {2020}
}
R2 v1 2026-06-23T15:02:56.748Z