Squareplus: A Softplus-Like Algebraic Rectifier
Machine Learning
2021-12-23 v1 Neural and Evolutionary Computing
Abstract
We present squareplus, an activation function that resembles softplus, but which can be computed using only algebraic operations: addition, multiplication, and square-root. Because squareplus is ~6x faster to evaluate than softplus on a CPU and does not require access to transcendental functions, it may have practical value in resource-limited deep learning applications.
Keywords
Cite
@article{arxiv.2112.11687,
title = {Squareplus: A Softplus-Like Algebraic Rectifier},
author = {Jonathan T. Barron},
journal= {arXiv preprint arXiv:2112.11687},
year = {2021}
}
Comments
https://github.com/jonbarron/squareplus