Related papers: Incorporating Local Step-Size Adaptivity into the …
For solving a broad class of nonconvex programming problems on an unbounded constraint set, we provide a self-adaptive step-size strategy that does not include line-search techniques and establishes the convergence of a generic approach…
We propose a location-adaptive self-normalization (SN) based test for change points in time series. The SN technique has been extensively used in change-point detection for its capability to avoid direct estimation of nuisance parameters.…
Tuning the step size of stochastic gradient descent is tedious and error prone. This has motivated the development of methods that automatically adapt the step size using readily available information. In this paper, we consider the family…
Gradient descent is slow to converge for ill-conditioned problems and non-convex problems. An important technique for acceleration is step-size adaptation. The first part of this paper contains a detailed review of step-size adaptation…
The computation time required by standard finite difference methods with fixed timesteps for solving fractional diffusion equations is usually very large because the number of operations required to find the solution scales as the square of…
Deep neural networks often develop spurious bias, reliance on correlations between non-essential features and classes for predictions. For example, a model may identify objects based on frequently co-occurring backgrounds rather than…
We consider various versions of adaptive Gibbs and Metropolis-within-Gibbs samplers, which update their selection probabilities (and perhaps also their proposal distributions) on the fly during a run by learning as they go in an attempt to…
The performance of Metropolis-Hastings algorithms is highly sensitive to the choice of step size, and miss-specification can lead to severe loss of efficiency. We study algorithms with randomized step sizes, considering both…
Dependency networks (Heckerman et al., 2000) provide a flexible framework for modeling complex systems with many variables by combining independently learned local conditional distributions through pseudo-Gibbs sampling. Despite their…
Step adjustment can improve the gait robustness of biped robots, however the adaptation of step timing is often neglected as it gives rise to non-convex problems when optimized over several footsteps. In this paper, we argue that it is not…
The stochastic volatility model is a popular tool for modeling the volatility of assets. The model is a nonlinear and non-Gaussian state space model, and consequently is difficult to fit. Many approaches, both classical and Bayesian, have…
Stochastic gradient descent is a canonical tool for addressing stochastic optimization problems, and forms the bedrock of modern machine learning and statistics. In this work, we seek to balance the fact that attenuating step-size is…
Over the past decade, many approaches have been introduced for traffic speed prediction. However, providing fine-grained, accurate, time-efficient, and adaptive traffic speed prediction for a growing transportation network where the size of…
A major challenge in current optimization research for deep learning is to automatically find optimal step sizes for each update step. The optimal step size is closely related to the shape of the loss in the update step direction. However,…
Visual instruction datasets from various distributors are released at different times and often contain a significant number of semantically redundant text-image pairs, depending on their task compositions (i.e., skills) or reference…
An Automated Sliced Gibbs framework is proposed for fully automated Markov chain Monte Carlo sampling from arbitrary finite dimensional probability kernels. The method targets unnormalized, non-smooth, heavy tailed, and highly multimodal…
In this paper, we present a novel approach to fitting mixture models based on estimating first the posterior distribution of the auxiliary variables that assign each observation to a group in the mixture. The posterior distributions of the…
This paper considers unconstrained convex optimization problems with time-varying objective functions. We propose algorithms with a discrete time-sampling scheme to find and track the solution trajectory based on prediction and correction…
Stein's method provides a way of bounding the distance of a probability distribution to a target distribution $\mu$. Here we develop Stein's method for the class of discrete Gibbs measures with a density $e^V$, where $V$ is the energy…
P-splines provide a flexible setting for modeling nonlinear model components based on a discretized penalty structure with a relatively simple computational backbone. Under a Bayesian inferential framework based on Markov chain Monte Carlo,…