Related papers: Convergence Results for Implicit--Explicit General…
Implicit regularization refers to the tendency of local search algorithms to converge to low-dimensional solutions, even when such structures are not explicitly enforced. Despite its ubiquity, the mechanism underlying this behavior remains…
Peer methods are a comprehensive class of time integrators offering numerous degrees of freedom in their coefficient matrices that can be used to ensure advantageous properties, e.g. A-stability or super-convergence. In this paper, we show…
In this manuscript, we address continuous unconstrained multi-objective optimization problems and we discuss descent type methods for the reconstruction of the Pareto set. Specifically, we analyze the class of Front Descent methods, which…
In-context learning (ICL) has shown impressive results in few-shot learning tasks, yet its underlying mechanism is still not fully understood. A recent line of work suggests that ICL performs gradient descent (GD)-based optimization…
The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper we propose some generalized CG (GCG) methods for solving the $\ell_1$-regularized…
In this paper, we study the generalized Douglas-Rachford algorithm and its cyclic variants which include many projection-type methods such as the classical Douglas-Rachford algorithm and the alternating projection algorithm. Specifically,…
Explicit stabilized integrators are an efficient alternative to implicit or semi-implicit methods to avoid the severe timestep restriction faced by standard explicit integrators applied to stiff diffusion problems. In this paper, we provide…
Just like the previous generation of task-tuned models, large language models (LLMs) that are adapted to tasks via prompt-based methods like in-context-learning (ICL) perform well in some setups but not in others. This lack of consistency…
This paper proposes a theoretical framework for establishing the energy dissipation of general implicit-explicit linear multistep methods (IMEX-LMMs) for gradient flows, by constructing a dissipative modified energy consisting of the…
We show that Newton's method converges globally at a linear rate for objective functions whose Hessians are stable. This class of problems includes many functions which are not strongly convex, such as logistic regression. Our linear…
In this paper, we consider a time-dependent discrete network model with highly varying connectivity. The approximation by time is performed using an implicit scheme. We propose the coarse scale approximation construction of network models…
In the purpose of making the consensus algorithm robust to outliers, consensus on the median value has recently attracted some attention. It has its applicability in for instance constructing a resilient distributed state estimator.…
This paper introduces a novel inexact gradient descent method with momentum (IGDm) considered as a general framework for various first-order methods with momentum. This includes, in particular, the inexact proximal point method (IPPm),…
Learning new tasks by drawing on prior experience gathered from other (related) tasks is a core property of any intelligent system. Gradient-based meta-learning, especially MAML and its variants, has emerged as a viable solution to…
We study the convergence rate of gradient-based local search methods for solving low-rank matrix recovery problems with general objectives in both symmetric and asymmetric cases, under the assumption of the restricted isometry property.…
The concept of asymptotically nonexpansive mappings is an important generalization of the class of nonexpansive mappings. Implicit midpoint procedures are extremely fundamental for solving equations involving nonlinear operators. This paper…
In this paper we continue the work on implicit-explicit (IMEX) time discretizations for the incompressible Oseen equations that we started in \cite{BGG23} (E. Burman, D. Garg, J. Guzm\`an, {\emph{Implicit-explicit time discretization for…
A non-uniform implicit-explicit L1 mixed finite element method (IMEX-L1-MFEM) is investigated for a class of time-fractional partial integro-differential equations (PIDEs) with space-time dependent coefficients and non-self-adjoint elliptic…
The optimization algorithms are crucial in training physics-informed neural networks (PINNs), as unsuitable methods may lead to poor solutions. Compared to the common gradient descent (GD) algorithm, implicit gradient descent (IGD)…
We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings,…