Related papers: Nonstationary iterative processes
Nesterov's well-known scheme for accelerating gradient descent in convex optimization problems is adapted to accelerating stationary iterative solvers for linear systems. Compared with classical Krylov subspace acceleration methods, the…
Planning problems where effects of actions are non-deterministic can be modeled as Markov decision processes. Planning problems are usually goal-directed. This paper proposes several techniques for exploiting the goal-directedness to…
This paper concerns models and convergence principles for dealing with stochasticity in a wide range of algorithms arising in nonlinear analysis and optimization in Hilbert spaces. It proposes a flexible geometric framework within which…
In appropriate frameworks, automatic differentiation is transparent to the user at the cost of being a significant computational burden when the number of operations is large. For iterative algorithms, implicit differentiation alleviates…
We analyse and explain the increased generalisation performance of iterate averaging using a Gaussian process perturbation model between the true and batch risk surface on the high dimensional quadratic. We derive three phenomena…
We propose a method for inference on moderately high-dimensional, nonlinear, non-Gaussian, partially observed Markov process models for which the transition density is not analytically tractable. Markov processes with intractable transition…
We present iterative solvers to approximate the solution of numerical schemes for stochastic Stefan problems. After briefly talking about the convergence results, we tackle the question of efficient strategies for solving the nonlinear…
We focus on the critical challenge of handling non-stationary data streams in online continual learning environments, where constrained storage capacity prevents complete retention of historical data, leading to catastrophic forgetting…
We present a simple yet powerful and applicable quadrature based scheme for constructing optimal iterative methods. According to the, still unproved, Kung-Traub conjecture an optimal iterative method based on $n+1$ evaluations could achieve…
This note considers the momentum method by Polyak and the accelerated gradient method by Nesterov, both without line search but with fixed step length applied to strictly convex quadratic functions assuming that exact gradients are used and…
We study a special case of the problem of statistical learning without the i.i.d. assumption. Specifically, we suppose a learning method is presented with a sequence of data points, and required to make a prediction (e.g., a classification)…
In studying the complexity of iterative processes it is usually assumed that the arithmetic operations of addition, multiplication, and division can be performed in certain constant times. This assumption is invalid if the precision…
We consider so-called $N$-fold integer programs (IPs) of the form $\max\{c^T x : Ax = b, \ell \leq x \leq u, x \in \mathbb Z^{nt}\}, where $A \in \mathbb Z^{(r+sn)\times nt} consists of $n$ arbitrary matrices $A^{(i)} \in \mathbb Z^{r\times…
In this paper, we propose a bootstrap method applied to massive data processed distributedly in a large number of machines. This new method is computationally efficient in that we bootstrap on the master machine without over-resampling,…
This paper presents a methodology for constructing iterative schemes of any order of convergence for solving nonlinear systems of equations. It also provides formulas for the order of convergence of any iterative schemes constructed using…
Fixed point iterations play a central role in the design and the analysis of a large number of optimization algorithms. We study a new iterative scheme in which the update is obtained by applying a composition of quasinonexpansive operators…
Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis.…
This paper studies nonstationary open dynamical systems from the statistical viewpoint. By open, we mean that trajectories may escape through holes in the phase space. By nonstationary, we mean that the dynamical model itself (as well as…
An accelerated class of adaptive scheme of iterative thresholding algorithms is studied analytically and empirically. They are based on the feedback mechanism of the null space tuning techniques (NST+HT+FB). The main contribution of this…
This paper develops a unified and computationally efficient method for change-point estimation along the time dimension in a non-stationary spatio-temporal process. By modeling a non-stationary spatio-temporal process as a piecewise…