Related papers: Modified Iterated Crank-Nicolson Method with Impro…
We study stochastic second-order methods for solving general non-convex optimization problems. We propose using a special version of momentum to stabilize the stochastic gradient and Hessian estimates in Newton's method. We show that…
The problem of numerical differentiation can be thought of as an inverse problem by considering it as solving a Volterra equation. It is well known that such inverse integral problems are ill-posed and one requires regularization methods to…
In this paper, we study the convergence rate of the DCA (Difference-of-Convex Algorithm), also known as the convex-concave procedure, with two different termination criteria that are suitable for smooth and nonsmooth decompositions…
Analytic and optimization methods for solving inverse kinematics (IK) problems have been deeply studied throughout the history of robotics. The two strategies have complementary strengths and weaknesses, but developing a unified approach to…
An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes $\mathcal{O}(\epsilon^{-3/2})$ iterations to drive the norm of the gradient of the objective function below a prescribed positive…
In this manuscript we propose and analyze an implicit two-point type method (or inertial method) for obtaining stable approximate solutions to linear ill-posed operator equations. The method is based on the iterated Tikhonov (iT) scheme. We…
Explicit stabilized methods are highly efficient time integrators for large and stiff systems of ordinary differential equations especially when applied to semi-discrete parabolic problems. However, when local spatial mesh refinement is…
We propose an iterative algorithm for low-rank matrix completion that can be interpreted as an iteratively reweighted least squares (IRLS) algorithm, a saddle-escaping smoothing Newton method or a variable metric proximal gradient method…
Transformers excel at in-context learning (ICL) -- learning from demonstrations without parameter updates -- but how they do so remains a mystery. Recent work suggests that Transformers may internally run Gradient Descent (GD), a…
We develop continuous-stage Runge-Kutta-Nystr\"Om (csRKN) methods in this paper. By leading weight function into the formalism of csRKN methods and modifying the original pattern of continuous-stage methods, we establish a new and larger…
We consider stochastic variational inequalities with monotone operators defined as the expected value of a random operator. We assume the feasible set is the intersection of a large family of convex sets. We propose a method that combines…
In this work, two Crank-Nicolson schemes without corrections are developed for sub-diffusion equations. First, we propose a Crank-Nicolson scheme without correction for problems with regularity assumptions only on the source term. Second,…
We investigate the problem of sequential linear data prediction for real life big data applications. The second order algorithms, i.e., Newton-Raphson Methods, asymptotically achieve the performance of the "best" possible linear data…
This paper proposes a second-order accurate numerical scheme for the Patlak-Keller-Segel system with various mobilities for the description of chemotaxis. Formulated in a variational structure, the entropy part is novelly discretized by a…
A method is proposed for solving equality constrained nonlinear optimization problems involving twice continuously differentiable functions. The method employs a trust funnel approach consisting of two phases: a first phase to locate an…
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. However, these algorithms might perform poorly especially if…
The systems of nonlinear Volterra integral equations of the first kind with jump discontinuous kernels are studied. The iterative numerical method for such nonlinear systems is proposed. Proposed method employs the modified…
An explicit second-order numerical method to integrate the isokinetic equations of motion is derived by fitting circular arcs through every three consecutive points of the discretized trajectory, so that the tangent and the curvature…
In this paper, we propose two second-order methods for solving the \(\ell_1\)-regularized composite optimization problem, which are developed based on two distinct definitions of approximate second-order stationary points. We introduce a…
Subspace clustering (SC) is a popular method for dimensionality reduction of high-dimensional data, where it generalizes Principal Component Analysis (PCA). Recently, several methods have been proposed to enhance the robustness of PCA and…