Related papers: Optimal Design and Analysis of the Exponentially W…
The ability to predict the behavior of a wireless channel in terms of the frame delivery ratio is quite valuable, and permits, e.g., to optimize the operating parameters of a wireless network at runtime, or to proactively react to the…
Simple exponential smoothing is widely used in forecasting economic time series. This is because it is quick to compute and it generally delivers accurate forecasts. On the other hand, its multivariate version has received little attention…
Zero-inflated models are frequently used to deal with data having many zeros. A commonly used model for over-dispersed data containing zeros is known as the zero-inflated Poisson model. However, to account for the heterogeneity of counts…
Wheelset intermittent over-creeps (WIOs), i.e., slips or slides, can decrease the overall traction and braking performance of Electric Multiple Units (EMUs). However, they are difficult to detect and isolate due to their small magnitude and…
Averaging, or smoothing, is a fundamental approach to obtain stable, de-noised estimates from noisy observations. In certain scenarios, observations made along trajectories of random dynamical systems are of particular interest. One popular…
Investigating the problem of setting control limits in the case of parameter uncertainty is more accessible when monitoring the variance because only one parameter has to be estimated. Simply ignoring the induced uncertainty frequently…
A multivariate dispersion control chart monitors changes in the process variability of multiple correlated quality characteristics. In this article, we investigate and compare the performance of charts designed to monitor variability based…
We present a real-time multivariate anomaly detection algorithm for data streams based on the Probabilistic Exponentially Weighted Moving Average (PEWMA). Our formulation is resilient to (abrupt transient, abrupt distributional, and gradual…
The Exponential Moving Average (EMA) is a cornerstone of widely used optimizers such as Adam. However, existing theoretical analyses of Adam-style methods have notable limitations: their guarantees can remain suboptimal in the zero-noise…
Many extensions and modifications have been made to standard process monitoring methods such as the exponentially weighted moving average (EWMA) chart and the cumulative sum (CUSUM) chart. In addition, new schemes have been proposed based…
Evolutionary algorithms (EAs) are general-purpose problem solvers that usually perform an unbiased search. This is reasonable and desirable in a black-box scenario. For combinatorial optimization problems, often more knowledge about the…
The exponential moving average (EMA) is a commonly used statistic for providing stable estimates of stochastic quantities in deep learning optimization. Recently, EMA has seen considerable use in generative models, where it is computed with…
Bandit Convex Optimization is a fundamental class of sequential decision-making problems, where the learner selects actions from a continuous domain and observes a loss (but not its gradient) at only one point per round. We study this…
Let \xi_0,\xi_1,...,\xi_{\omega-1} be observations from the hidden Markov model with probability distribution P^{\theta_0}, and let \xi_{\omega},\xi_{\omega+1},... be observations from the hidden Markov model with probability distribution…
We examine two different techniques for parameter averaging in GAN training. Moving Average (MA) computes the time-average of parameters, whereas Exponential Moving Average (EMA) computes an exponentially discounted sum. Whilst MA is known…
The paper addresses a sequential changepoint detection problem, assuming that the duration of change may be finite and unknown. This problem is of importance for many applications, e.g., for signal and image processing where signals appear…
The scaling of the optimal AdamW weight decay hyperparameter with model and dataset size is critical as we seek to build larger models, but is poorly understood. We show that weights learned by AdamW can be understood as an exponential…
In this work, we offer a theoretical analysis of two modern optimization techniques for training large and complex models: (i) adaptive optimization algorithms, such as Adam, and (ii) the model exponential moving average (EMA).…
We argue against the use of generally weighted moving average (GWMA) control charts. Our primary reasons are the following: 1) There is no recursive formula for the GWMA control chart statistic, so all previous data must be stored and used…
We consider the simple changepoint problem setting, where observations are independent, iid pre-change and iid post-change, with known pre- and post-change distributions. The Shiryaev-Roberts detection procedure is known to be…