Related papers: Adaptive time step selection for Spectral Deferred…
A sequential quadratic programming method is designed for solving general smooth nonlinear stochastic optimization problems subject to expectation equality constraints. We consider the setting where the objective and constraint function…
Existing asynchronous distributed optimization algorithms often use diminishing step-sizes that cause slow practical convergence, or use fixed step-sizes that depend on and decrease with an upper bound of the delays. Not only are such delay…
This paper proposes a novel approach to adaptive step sizes in stochastic gradient descent (SGD) by utilizing quantities that we have identified as numerically traceable -- the Lipschitz constant for gradients and a concept of the local…
This paper presents a sequence of deferred correction (DC) schemes built recursively from the implicit midpoint scheme for the numerical solution of general first order ordinary differential equations (ODEs). It is proven that each scheme…
We present an adaptive algorithm for effectively solving rough differential equations (RDEs) using the log-ODE method. The algorithm is based on an error representation formula that accurately describes the contribution of local errors to…
We propose a stochastic optimization method for minimizing loss functions, expressed as an expected value, that adaptively controls the batch size used in the computation of gradient approximations and the step size used to move along such…
With the fast development of deep learning, it has become common to learn big neural networks using massive training data. Asynchronous Stochastic Gradient Descent (ASGD) is widely adopted to fulfill this task for its efficiency, which is,…
In this work we examine blind adaptive and iterative decision feedback (DF) receivers for direct sequence code division multiple access (DS-CDMA) systems in frequency selective channels. Code-constrained minimum variance (CMV) and constant…
In many applications, gradient evaluations are inherently approximate, motivating the development of optimization methods that remain reliable under inexact first-order information. A common strategy in this context is adaptive evaluation,…
Block coordinate descent (BCD) methods and their variants have been widely used in coping with large-scale nonconstrained optimization problems in many fields such as imaging processing, machine learning, compress sensing and so on. For…
We propose a new method of adaptive piecewise approximation based on Sinc points for ordinary differential equations. The adaptive method is a piecewise collocation method which utilizes Poly-Sinc interpolation to reach a preset level of…
The difference-of-convex algorithm (DCA) and its variants are the most popular methods to solve the difference-of-convex optimization problem. Each iteration of them is reduced to a convex optimization problem, which generally needs to be…
Spatially-coupled (SC) codes are a class of low-density parity-check (LDPC) codes that have excellent performance thanks to the degrees of freedom they offer. An SC code is designed by partitioning a base matrix into components, the number…
Efficient high order numerical methods for evolving the solution of an ordinary differential equation are widely used. The popular Runge--Kutta methods, linear multi-step methods, and more broadly general linear methods, all have a global…
Estimation and counterfactual experiments in dynamic discrete choice models with large state spaces pose computational difficulties. This paper proposes a model-adaptive approach, based on the conjugate gradient (CG) method, to solve the…
We consider the difference of convex (DC) optimization problem subject to box constraints. Utilizing epsilon-subdifferentials of DC components of the objective, we develop a new method for finding global solutions to this problem. The…
Dual Coordinate Descent (DCD) and Block Dual Coordinate Descent (BDCD) are important iterative methods for solving convex optimization problems. In this work, we develop scalable DCD and BDCD methods for the kernel support vector machines…
Stochastic gradient descent (SGD) type optimization schemes are fundamental ingredients in a large number of machine learning based algorithms. In particular, SGD type optimization schemes are frequently employed in applications involving…
The pseudospectra of a linear time-invariant system are the sets in the complex plane consisting of all the roots of the characteristic equation when the system matrices are subjected to all possible perturbations with a given upper bound.…
Neural network-based solvers for partial differential equations (PDEs) have attracted considerable attention, yet they often face challenges in accuracy and computational efficiency. In this work, we focus on time-dependent PDEs and observe…