Related papers: Applying Lepskij-Balancing in Practice

This paper studies gradient dynamics subject to additive random noise, which may arise from sources such as stochastic gradient estimation, measurement noise, or stochastic sampling errors. To analyze the robustness of such stochastic…

The idea of exploiting sparseness in under-determined damage characterization problems is not new, and regularizations techniques that tend to promote sparseness, such as L1-norm minimization, have been investigated in the last ten years or…

The $\chi^2$-principle generalizes the Morozov discrepancy principle (MDP) to the augmented residual of the Tikhonov regularized least squares problem. Weighting of the data fidelity by a known Gaussian noise distribution on the measured…

The objective of this work is to quantify the reconstruction error in sparse inverse problems with measures and stochastic noise, motivated by optimal sensor placement. To be useful in this context, the error quantities must be explicit in…

In this paper we focus on the parameter estimation of dynamic load models with stochastic terms, in particular, load models where protection settings are uncertain, such as in aggregated air conditioning units. We show how the uncertainty…

This paper presents a bias-variance tradeoff of graph Laplacian regularizer, which is widely used in graph signal processing and semi-supervised learning tasks. The scaling law of the optimal regularization parameter is specified in terms…

The ranking problem is to order a collection of units by some unobserved parameter, based on observations from the associated distribution. This problem arises naturally in a number of contexts, such as business, where we may want to rank…

Concerning Numerical Stochastic Perturbation Theory, we discuss the convergence of the stochastic process (idea of the proof, features of the limit distribution, rate of convergence to equilibrium). Then we also discuss the expected…

This paper presents a detailed theoretical analysis of the three stochastic approximation proximal gradient algorithms proposed in our companion paper [49] to set regularization parameters by marginal maximum likelihood estimation. We prove…

There is growing body of learning problems for which it is natural to organize the parameters into matrix, so as to appropriately regularize the parameters under some matrix norm (in order to impose some more sophisticated prior knowledge).…

In high-dimensional and/or non-parametric regression problems, regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure…

As machine learning applications grow increasingly ubiquitous and complex, they face an increasing set of requirements beyond accuracy. The prevalent approach to handle this challenge is to aggregate a weighted combination of requirement…

This paper concerns parameterized convex infinite (or semi-infinite) inequality systems whose decision variables run over general infinite-dimensional Banach (resp. finite-dimensional) spaces and that are indexed by an arbitrary fixed set T…

Stochastic approximation is a powerful class of algorithms with celebrated success. However, a large body of previous analysis focuses on stochastic approximations driven by contractive operators, which is not applicable in some important…

Standard regularization methods that are used to compute solutions to ill-posed inverse problems require knowledge of the forward model. In many real-life applications, the forward model is not known, but training data is readily available.…

Linear Least Squares is a very well known technique for parameter estimation, which is used even when sub-optimal, because of its very low computational requirements and the fact that exact knowledge of the noise statistics is not required.…

We stabilize a prescribed cycle or an equilibrium of the difference equation using pulsed stochastic control. Our technique, inspired by the Kolmogorov's Law of Large Numbers, activates a stabilizing effect of stochastic perturbation and…

This paper offers a qualitative insight into the convergence of Bayesian parameter inference in a setup which mimics the modeling of the spread of a disease with associated disease measurements. Specifically, we are interested in the…

Within Bayesian state estimation, considerable effort has been devoted to incorporating constraints into state estimation for process optimization, state monitoring, fault detection and control. Nonetheless, in the domain of state-space…

This paper focuses on investigating an inexact stochastic model-based optimization algorithm that integrates preconditioning techniques for solving stochastic composite optimization problems. The proposed framework unifies and extends the…