Related papers: Quantifying the Computational Advantage of Forward…

Stochastic gradient descent (SGD) is a fundamental optimization algorithm widely used in modern machine learning. In this paper, we propose Factor-Augmented SGD (FSGD), a new optimization method that leverages latent factor representations…

The design of microfluidic devices is a cumbersome and tedious process that can be significantly improved by simulation. Methods based on Computational Fluid Dynamics (CFD) are considered state-of-the-art, but require extensive compute time…

We present a previously unexplored forward-mode differentiation method for Maxwell's equations, with applications in the field of sensitivity analysis. This approach yields exact gradients and is similar to the popular adjoint variable…

Hoffman's classical result gives a bound on the distance of a point from a convex and compact polytope in terms of the magnitude of violation of the constraints. Recently, several results showed that Hoffman's bound can be used to derive…

The Fourier-Galerkin method (in short FFTH) has gained popularity in numerical homogenisation because it can treat problems with a huge number of degrees of freedom. Because the method incorporates the fast Fourier transform (FFT) in the…

In this work, we benchmark and discuss the performance of the scalable methods for the Poisson problem which are used widely in practice: the fast Fourier transform (FFT), the fast multipole method (FMM), the geometric multigrid (GMG), and…

We demonstrate a new computational illumination technique that achieves large space-bandwidth-time product, for quantitative phase imaging of unstained live samples in vitro. Microscope lenses can have either large field of view (FOV) or…

Stochastic processes play a fundamental role in physics, mathematics, engineering and finance. One potential application of quantum computation is to better approximate properties of stochastic processes. For example, quantum algorithms for…

Second-order optimization techniques have the potential to achieve faster convergence rates compared to first-order methods through the incorporation of second-order derivatives or statistics. However, their utilization in deep learning is…

The graph Fourier transform (GFT) is in general dense and requires O(n^2) time to compute and O(n^2) memory space to store. In this paper, we pursue our previous work on the approximate fast graph Fourier transform (FGFT). The FGFT is…

Spectral analysis provides one of the most effective paradigms for information-preserving dimensionality reduction, as simple descriptions of naturally occurring signals are often obtained via few terms of periodic basis functions. In this…

We consider gradient descent (GD) with a constant stepsize applied to logistic regression with linearly separable data, where the constant stepsize $\eta$ is so large that the loss initially oscillates. We show that GD exits this initial…

The Fast Multipole Method (FMM) for the Poisson equation is extended to the case of non-axisymmetric problems in an axisymmetric domain, described by cylindrical coordinates. The method is based on a Fourier decomposition of the source into…

In this paper, we consider an unconstrained stochastic optimization problem where the objective function exhibits high-order smoothness. Specifically, we propose a new stochastic first-order method (SFOM) with multi-extrapolated momentum,…

Finite-difference time-domain (FDTD) simulations often involve physical quantities spanning multiple orders of magnitude, such as the speed of light or electromagnetic field amplitudes. The standard practice for maintaining numerical…

Temporal-Difference (TD) learning with nonlinear smooth function approximation for policy evaluation has achieved great success in modern reinforcement learning. It is shown that such a problem can be reformulated as a stochastic…

In this paper, a reduced-order model (ROM) based on the proper orthogonal decomposition and the discrete empirical interpolation method is proposed for efficiently simulating time-fractional partial differential equations (TFPDEs). Both…

Inference in models where the parameter is defined by moment inequalities is of interest in many areas of economics. This paper develops a new method for improving the performance of generalized moment selection (GMS) testing procedures in…

We propose a strategy to achieve the fastest convergence in the Wang-Landau algorithm with varying modification factors. With this strategy, the convergence of a simulation is at least as good as the conventional Monte Carlo algorithm, i.e.…

This paper proposes a novel Generalized Non-Standard Finite Difference (GNSFD) scheme for the numerical solution of a class of fractional partial differential equations (FrPDEs). The formulation of the method is grounded in optimization and…