Related papers: Optimization of Discrete Parameters Using the Adap…

Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter,…

A novel algorithm is proposed to solve the sample-based optimal transport problem. An adversarial formulation of the push-forward condition uses a test function built as a convolution between an adaptive kernel and an evolving probability…

We develop a novel iterative algorithm for locally optimal experimental design under constraints, like budget or performance constraints. It is an adaptive discretization algorithm. In every iteration, a discretized version of the…

Stochastic optimization problems often involve data distributions that change in reaction to the decision variables. This is the case for example when members of the population respond to a deployed classifier by manipulating their features…

This paper proposes a new gradient-based optimization approach for designing optimal feedback kernels for parabolic distributed parameter systems with boundary control. Unlike traditional kernel optimization methods for parabolic systems,…

We consider an adaptive finite element method with arbitrary but fixed polynomial degree $p \ge 1$, where adaptivity is driven by an edge-based residual error estimator. Based on the modified maximum criterion from [Diening et al, Found.…

We systematically develop a learning-based treatment of stochastic optimal control (SOC), relying on direct optimization of parametric control policies. We propose a derivation of adjoint sensitivity results for stochastic differential…

Selecting the most relevant or informative features is a key issue in actual machine learning problems. Since an exhaustive search is not feasible even for a moderate number of features, an intelligent search strategy must be employed for…

Gradient compression is of growing interests for solving constrained optimization problems including compressed sensing, noisy recovery and matrix completion under limited communication resources and storage costs. Convergence analysis of…

We consider a class of convex optimization problems over the simplex of probability measures. Our framework comprises optimal experimental design (OED) problems, in which the measure over the design space indicates which experiments are…

Differentially private stochastic gradient descent (DP-SGD) is broadly considered to be the gold standard for training and fine-tuning neural networks under differential privacy (DP). With the increasing availability of high-quality…

Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering (ODQA) systems. However, current AC approaches require tuning of all model parameters, and training state-of-the-art ODQA…

Biological populations are subject to fluctuating environmental conditions. Different adaptive strategies can allow them to cope with these fluctuations: specialization to one particular environmental condition, adoption of a generalist…

Motivation: Estimating parameters from data is a key stage of the modelling process, particularly in biological systems where many parameters need to be estimated from sparse and noisy data sets. Over the years, a variety of heuristics have…

In this paper, a gradient-free distributed algorithm is introduced to solve a set constrained optimization problem under a directed communication network. Specifically, at each time-step, the agents locally compute a so-called…

We are interested in modelling Darwinian evolution, resulting from the interplay of phenotypic variation and natural selection through ecological interactions. Our models are rooted in the microscopic, stochastic description of a population…

We propose a flexible gradient tracking approach with adjustable computation and communication steps for solving distributed stochastic optimization problem over networks. The proposed method allows each node to perform multiple local…

In this paper, we propose a conditional gradient method for solving constrained vector optimization problems with respect to a partial order induced by a closed, convex and pointed cone with nonempty interior. When the partial order under…

We analyze the convergence of gradient-based optimization algorithms that base their updates on delayed stochastic gradient information. The main application of our results is to the development of gradient-based distributed optimization…

This article investigates a distributed aggregative optimization problem subject to coupled affine inequality constraints, in which local objective functions depend not only on their own decision variables but also on an aggregation of all…