Related papers: Pt\'ak's nondiscrete induction and its application…
We study the rate of convergence of linear two-time-scale stochastic approximation methods. We consider two-time-scale linear iterations driven by i.i.d. noise, prove some results on their asymptotic covariance and establish asymptotic…
Iterative refinement -- start with a random guess, then iteratively improve the guess -- is a useful paradigm for representation learning because it offers a way to break symmetries among equally plausible explanations for the data. This…
We consider an equation of multiple variables in which a partial derivative does not vanish at a point. The implicit function theorem provides a local existence and uniqueness of the function for the equation. In this paper, we propose an…
We study composite optimization problems in which the smooth part of the objective function is \( p \)-times continuously differentiable, where \( p \geq 1 \) is an integer. Higher-order methods are known to be effective for solving such…
We present a new rational approximation algorithm based on the empirical interpolation method for interpolating a family of parametrized functions to rational polynomials with invariant poles, leading to efficient numerical algorithms for…
We study accelerated optimization methods in the Gaussian phase retrieval problem. In this setting, we prove that gradient methods with Polyak or Nesterov momentum have similar implicit regularization to gradient descent. This implicit…
In present paper we propose seemingly new method for finding solutions of some types of nonlinear PDEs in closed form. The method is based on decomposition of nonlinear operators on sequence of operators of lower orders. It is shown that…
Based on the computation of a superset of the implicit support, implicitization of a parametrically given hyper-surface is reduced to computing the nullspace of a numeric matrix. Our approach exploits the sparseness of the given parametric…
This paper proposes an implicit family of sub-step integration algorithms grounded in the explicit singly diagonally implicit Runge-Kutta (ESDIRK) method. The proposed methods achieve third-order consistency per sub-step and thus the…
We consider the problem of low-rank decomposition of incomplete multiway tensors. Since many real-world data lie on an intrinsically low dimensional subspace, tensor low-rank decomposition with missing entries has applications in many data…
Numerical solutions for flows in partially saturated porous media pose challenges related to the non-linearity and elliptic-parabolic degeneracy of the governing Richards' equation. Iterative methods are therefore required to manage the…
We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings,…
In this paper, we propose an inexact proximal Newton-type method for nonconvex composite problems. We establish the global convergence rate of the order $\mathcal{O}(k^{-1/2})$ in terms of the minimal norm of the KKT residual mapping and…
Through introducing a new iterative formula for divided differnce using Neville's and Aitken's algorithms,we study new iterative methods for interpolation,numerical differentiation and numerical integration formulas with arbitrary order of…
Interpretability has become an important issue in the machine learning field, along with the success of layered neural networks in various practical tasks. Since a trained layered neural network consists of a complex nonlinear relationship…
In this pedagogical article, we present a simple direct matrix method for analytically computing the Jacobian of nonlinear algebraic equations that arise from the discretization of nonlinear integro-differential equations. The method is…
It is important to understand how dropout, a popular regularization method, aids in achieving a good generalization solution during neural network training. In this work, we present a theoretical derivation of an implicit regularization of…
In this manuscript, an original numerical procedure for the nonlinear peridynamics on arbitrarily--shaped two-dimensional (2D) closed manifolds is proposed. When dealing with non parameterized 2D manifolds at the discrete scale, the problem…
Iterative regularization exploits the implicit bias of an optimization algorithm to regularize ill-posed problems. Constructing algorithms with such built-in regularization mechanisms is a classic challenge in inverse problems but also in…
A decomposition principle for nonlinear dynamic compartmental systems is introduced in the present paper. This theory is based on the mutually exclusive and exhaustive, analytical and dynamic, novel system and subsystem partitioning…