Nonsmooth Optimisation and neural networks
Optimization and Control
2025-03-05 v1
Abstract
In this paper, we study neural networks from the point of view of nonsmooth optimisation, namely, quasidifferential calculus. We restrict ourselves to the case of uniform approximation by a neural network without hidden layers, the activation functions are restricted to continuous strictly increasing functions. We develop an algorithm for computing the approximation with one hidden layer through a step-by-step procedure. The nonsmooth analysis techniques demonstrated their efficiency. In particular, they partially explain why the developed step-by-step procedure may run without any objective function improvement after just one step of the procedure.
Cite
@article{arxiv.2503.01860,
title = {Nonsmooth Optimisation and neural networks},
author = {Vinesha Peiris and Nadezda Sukhorukova},
journal= {arXiv preprint arXiv:2503.01860},
year = {2025}
}