Related papers: Deep learning as optimal control problems: models …

In this work, we consider the two dimensional tidal dynamics equations in a bounded domain and address some optimal control problems like total energy minimization, minimization of dissipation of energy of the flow, etc. We also examine an…

Deep neural networks (DNNs) are powerful machine learning models and have succeeded in various artificial intelligence tasks. Although various architectures and modules for the DNNs have been proposed, selecting and designing the…

Distributed optimization is the standard way of speeding up machine learning training, and most of the research in the area focuses on distributed first-order, gradient-based methods. Yet, there are settings where some…

In this work, we investigate a neural network based solver for optimal control problems (without / with box constraint) for linear and semilinear second-order elliptic problems. It utilizes a coupled system derived from the first-order…

Using an optimization algorithm to solve a machine learning problem is one of mainstreams in the field of science. In this work, we demonstrate a comprehensive comparison of some state-of-the-art first-order optimization algorithms for…

We consider the variational discretization of a linear-quadratic optimal control problem with pointwise control and state constraints. In order to allow for a Fr\'echet smooth norm, the problem is reformulated by means of a reflexive…

In [1] we consider an optimal control problem subject to a semilinear elliptic PDE together with its variational discretization, where we provide a condition which allows to decide whether a solution of the necessary first order conditions…

The conjoining of dynamical systems and deep learning has become a topic of great interest. In particular, neural differential equations (NDEs) demonstrate that neural networks and differential equation are two sides of the same coin.…

Deep learning has received considerable empirical successes in recent years. However, while many ad hoc tricks have been discovered by practitioners, until recently, there has been a lack of theoretical understanding for tricks invented in…

The solution of linear inverse problems arising, for example, in signal and image processing is a challenging problem since the ill-conditioning amplifies, in the solution, the noise present in the data. Recently introduced algorithms based…

As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow…

Oftentimes, machine learning applications using neural networks involve solving discrete optimization problems, such as in pruning, parameter-isolation-based continual learning and training of binary networks. Still, these discrete problems…

We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex and the the variable control has two components, the first being absolutely continuous and the second singular. The system is…

Machine learning pipelines often rely on optimization procedures to make discrete decisions (e.g., sorting, picking closest neighbors, or shortest paths). Although these discrete decisions are easily computed, they break the…

We consider a broker who has to place a large order which consumes a sizable part of average daily trading volume. The broker's aim is thus to minimize execution costs he incurs from the adverse impact of his trades on market prices. By…

Optimization is an integral part of modern deep learning. Recently, the concept of learned optimizers has emerged as a way to accelerate this optimization process by replacing traditional, hand-crafted algorithms with meta-learned…

A classic approach for solving differential equations with neural networks builds upon neural forms, which employ the differential equation with a discretisation of the solution domain. Making use of neural forms for time-dependent…

Deep learning achieves remarkable generalization capability with overwhelming number of model parameters. Theoretical understanding of deep learning generalization receives recent attention yet remains not fully explored. This paper…

Deep learning has arguably achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex dependency between input features and labels. While neural networks…

The theory of deep learning focuses almost exclusively on supervised learning, non-convex optimization using stochastic gradient descent, and overparametrized neural networks. It is common belief that the optimizer dynamics, network…