Differentiable Spline Approximations
Abstract
The paradigm of differentiable programming has significantly enhanced the scope of machine learning via the judicious use of gradient-based optimization. However, standard differentiable programming methods (such as autodiff) typically require that the machine learning models be differentiable, limiting their applicability. Our goal in this paper is to use a new, principled approach to extend gradient-based optimization to functions well modeled by splines, which encompass a large family of piecewise polynomial models. We derive the form of the (weak) Jacobian of such functions and show that it exhibits a block-sparse structure that can be computed implicitly and efficiently. Overall, we show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications such as image segmentation, 3D point cloud reconstruction, and finite element analysis.
Cite
@article{arxiv.2110.01532,
title = {Differentiable Spline Approximations},
author = {Minsu Cho and Aditya Balu and Ameya Joshi and Anjana Deva Prasad and Biswajit Khara and Soumik Sarkar and Baskar Ganapathysubramanian and Adarsh Krishnamurthy and Chinmay Hegde},
journal= {arXiv preprint arXiv:2110.01532},
year = {2021}
}
Comments
9 pages, accepted in Neurips 2021