Related papers: Towards Quantifying the Preconditioning Effect of …
The emergence of big data has caused a dramatic shift in the operating regime for optimization algorithms. The performance bottleneck, which used to be computations, is now often communications. Several gradient compression techniques have…
While stochastic gradient descent (SGD) is still the most popular optimization algorithm in deep learning, adaptive algorithms such as Adam have established empirical advantages over SGD in some deep learning applications such as training…
Adaptive first-order optimizers are fundamental tools in deep learning, although they may suffer from poor generalization due to the nonuniform gradient scaling. In this work, we propose AdamL, a novel variant of the Adam optimizer, that…
We review the $N$-Body Problem in arbitrary dimension $d$ at the kinematical level, with modelling Background Independence in mind. In particular, we give a structural analysis of its reduced configuration spaces, decomposing this subject…
In this paper, we test the performance of some modern stochastic optimization methods and practices in application to digital pre-distortion problem, that is a valuable part of processing signal on base stations providing wireless…
This article is concerned with the question of constructing effcient multigrid preconditioners for the linear systems arising when applying semismooth Newton methods to large-scale linear-quadratic optimization problems constrained by…
Analog quantum simulations---simulations of one Hamiltonian by another---is one of the major goals in the noisy intermediate-scale quantum computation (NISQ) era, and has many applications in quantum complexity. We initiate the rigorous…
In this paper we analyze the relaxed form of a shape optimization problem with state equation $\{{array}{ll} -div \big(a(x)Du\big)=f\qquad\hbox{in}D \hbox{boundary conditions on}\partial D. {array}.$ The new fact is that the term $f$ is…
We propose a perturbative approach to determine the time-dependent Dyson map and the metric operator associated with time-dependent non-Hermitian Hamiltonians. We apply the method to a pair of explicitly time-dependent two dimensional…
We propose Adam-SHANG, a Lyapunov-guided Adam-type method that couples momentum, adaptive preconditioning, and a curvature-aware correction through a more stable lagged-preconditioner update. For stochastic smooth convex optimization, we…
The Adam optimizer remains one of the most widely used optimizers in deep learning, and effectively tuning its hyperparameters is key to optimizing performance. However, tuning can be tedious and costly. Freeze-thaw Bayesian Optimization…
We consider the Allen-Cahn equation with the so-called truncated Laplacians, which are fully nonlinear differential operators that depend on some eigenvalues of the Hessian matrix. By monitoring the sign of a quantity that is responsible…
A new and intuitive perturbative approach to time-dependent quantum mechanics problems is presented, which is useful in situations where the evolution of the Hamiltonian is slow. The state of a system which starts in an instantaneous…
This paper is concerned with the sensitivity analysis of a class of parameterized fixed-point problems that arise in the context of obstacle-type quasi-variational inequalities. We prove that, if the operators in the considered fixed-point…
We study the quadratic form associated to the kinetic energy operator in the presence of an external magnetic field in d = 3. We show that if the radial component of the magnetic field does not vanish identically, then the classical lower…
In this paper we consider the problem u in H^1_0 (Omega), - div (A(x) Du) = H(x, u, Du) + f(x) + a_0 (x) u in D'(Omega), where Omega is an open bounded set of R^N, N \geq 3, A(x) is a coercive matrix with coefficients in L^\infty(Omega),…
A new domain decomposition preconditioner is introduced for efficiently solving linear systems Ax = b with a symmetric positive definite matrix A. The particularity of the new preconditioner is that it is not necessary to have access to the…
There is growing awareness that errors in the model equations cannot be ignored in data assimilation methods such as four-dimensional variational assimilation (4D-Var). If allowed for, more information can be extracted from observations,…
A new preconditioner is developed for high order finite element approximation of linear elastic problems on triangular meshes in two dimensions. The new preconditioner results in a condition number that is bounded independently of the…
The de Haas - van Alphen effect in quasi-two-dimensional metals is studied at arbitrary parameters. The oscillations of the chemical potential may substantially change the temperature dependence of harmonic amplitudes that is usually used…