Related papers: Spurious minimizers in non uniform Fourier samplin…
A systematic procedure for optimising the friction coefficient in underdamped Langevin dynamics as a sampling tool is given by taking the gradient of the associated asymptotic variance with respect to friction. We give an expression for…
In this work we investigate the practicality of stochastic gradient descent and recently introduced variants with variance-reduction techniques in imaging inverse problems. Such algorithms have been shown in the machine learning literature…
We discuss non-Euclidean deterministic and stochastic algorithms for optimization problems with strongly and uniformly convex objectives. We provide accuracy bounds for the performance of these algorithms and design methods which are…
The (constrained) minimization of a ratio of set functions is a problem frequently occurring in clustering and community detection. As these optimization problems are typically NP-hard, one uses convex or spectral relaxations in practice.…
The problem of computing the Fourier Transform of a signal whose spectrum is dominated by a small number $k$ of frequencies quickly and using a small number of samples of the signal in time domain (the Sparse FFT problem) has received…
Best subset selection is considered the `gold standard' for many sparse learning problems. A variety of optimization techniques have been proposed to attack this non-convex and NP-hard problem. In this paper, we investigate the dual forms…
In this paper an approach for decreasing the computational effort required for the split-step Fourier method (SSFM) is introduced. It is shown that using the sparsity property of the simulated signals, the compressive sampling algorithm can…
In this paper, we consider a class of structured nonsmooth fractional minimization, where the first part of the objective is the ratio of a nonnegative nonsmooth nonconvex function to a nonnegative nonsmooth convex function, while the…
Applications such as Magnetic Resonance Tomography acquire imaging data by point samples of their Fourier transform. This raises the question of balancing the efficiency of the sampling strategies with the approximation accuracy of an…
We consider the problem of finding the minimizer of a convex function $F: \mathbb R^d \rightarrow \mathbb R$ of the form $F(w) := \sum_{i=1}^n f_i(w) + R(w)$ where a low-rank factorization of $\nabla^2 f_i(w)$ is readily available. We…
This paper studies the sparse identification problem of unknown sparse parameter vectors in stochastic dynamic systems. Firstly, a novel sparse identification algorithm is proposed, which can generate sparse estimates based on least squares…
We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks. We propose stochastic approximation algorithms which optimize a series of regularized, nonlinearized losses on large…
We construct importance sampling schemes for stochastic differential equations with small noise and fast oscillating coefficients. Standard Monte Carlo methods perform poorly for these problems in the small noise limit. With multiscale…
The prevalence of variational methods in near-term quantum computing makes optimizer choice critical, yet selection is frequently intuition-based. We therefore present a systematic benchmark of eight classical optimization algorithms for…
We propose a novel sparse spectrum approximation of Gaussian process (GP) tailored for Bayesian optimization. Whilst the current sparse spectrum methods provide desired approximations for regression problems, it is observed that this…
In this paper, the optimal sampling strategies (uniform or nonuniform) and distortion tradeoffs for Gaussian bandlimited periodic signals with additive white Gaussian noise are studied. Our emphasis is on characterizing the optimal sampling…
Simulating a Gaussian process requires sampling from a high-dimensional Gaussian distribution, which scales cubically with the number of sample locations. Spectral methods address this challenge by exploiting the Fourier representation,…
Signal processing is rich in inherently continuous and often nonlinear applications, such as spectral estimation, optical imaging, and super-resolution microscopy, in which sparsity plays a key role in obtaining state-of-the-art results.…
In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We introduce a family of stochastic algorithms that can handle a…
We consider stochastic variational inequality problems where the mapping is monotone over a compact convex set. We present two robust variants of stochastic extragradient algorithms for solving such problems. Of these, the first scheme…