Related papers: Analysis for the Slow Convergence in Arimoto Algor…
Given an irrational vector $\alpha$ in $\mathbb{R}^{d}$, a continuous function $f(x)$ on the torus $\mathbb{T}^{d}$ and suitable weights $\Phi(N,n)$ such that $\sum_{n=-\infty}^{+\infty}\Phi(N,n)=1$, we estimate the speed of convergence to…
In this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a Hilbert space of functions. We consider and analyze published procedures, both…
In recent years, functional linear models have attracted growing attention in statistics and machine learning, with the aim of recovering the slope function or its functional predictor. This paper considers online regularized learning…
In recent studies on sparse modeling, the nonconvex regularization approaches (particularly, $L_{q}$ regularization with $q\in(0,1)$) have been demonstrated to possess capability of gaining much benefit in sparsity-inducing and efficiency.…
This paper proposes a two-point inertial proximal point algorithm to find zero of maximal monotone operators in Hilbert spaces. We obtain weak convergence results and non-asymptotic $O(1/n)$ convergence rate of our proposed algorithm in…
In this paper we make a theoretical analysis of the convergence rates of Kaczmarz and Extended Kaczmarz projection algorithms for some of the most practically used control sequences. We first prove an at least linear convergence rate for…
In this letter, an accelerated quadratic programming (QP) algorithm is proposed based on the proximal gradient method. The algorithm can achieve convergence rate $O(1/p^{\alpha})$, where $p$ is the iteration number and $\alpha$ is the given…
Convergence of the Kalman filter is best analyzed by studying the contraction of the Riccati map in the space of positive definite (covariance) matrices. In this paper, we explore how this contraction property relates to a more fundamental…
The Kullback-Leibler (KL) divergence plays a central role in probabilistic machine learning, where it commonly serves as the canonical loss function. Optimization in such settings is often performed over the probability simplex, where the…
We consider optimization algorithms that successively minimize simple Taylor-like models of the objective function. Methods of Gauss-Newton type for minimizing the composition of a convex function and a smooth map are common examples. Our…
In this paper, we establish sublinear and linear convergence of fixed point iterations generated by averaged operators in a Hilbert space. Our results are achieved under a bounded H\"older regularity assumption which generalizes the…
In this paper we consider parallelization for applications whose objective can be expressed as maximizing a non-monotone submodular function under a cardinality constraint. Our main result is an algorithm whose approximation is arbitrarily…
We study a class of finite state machines, called \defi{$w$-matching machines}, which yield to simulate the behavior of pattern matching algorithms while searching for a pattern $w$. They can be used to compute the asymptotic speed, i.e.…
Expectation maximization (EM) is the default algorithm for fitting probabilistic models with missing or latent variables, yet we lack a full understanding of its non-asymptotic convergence properties. Previous works show results along the…
Consider the problem of estimating the median of N items to a precision epsilon, i.e., the estimate should be such that, with a high probability, the number of items, with values both smaller than and larger than this estimate, is less than…
Given $N\ge2$ closed subspaces $M_1,\dotsc, M_N$ of a Hilbert space $X$, let $P_k$ denote the orthogonal projection onto $M_k$, $1\le k\le N$. It is known that the sequence $(x_n)$, defined recursively by $x_0=x$ and $x_{n+1}=P_N\cdots…
We propose first order algorithms for convex optimization problems where the feasible set is described by a large number of convex inequalities that is to be explored by subgradient projections. The first algorithm is an adaptation of a…
This article derives lower bounds on the convergence rate of continuous-time gradient-based optimization algorithms. The algorithms are subjected to a time-normalization constraint that avoids a reparametrization of time in order to make…
The Kuramoto model, which describes synchronization phenomena, is a system of ordinary differential equations on $N$-torus defined as coupled harmonic oscillators. The order parameter is often used to measure the degree of synchronization.…
For a regression model, we consider the risk of the maximum likelihood estimator with respect to $\alpha$-divergence, which includes the special cases of Kullback-Leibler divergence, Hellinger distance and $\chi^2$ divergence. The…