Related papers: Anytime Tail Averaging
Tail Averaging improves on Polyak averaging's non-asymptotic behaviour by excluding a number of leading iterates of stochastic optimization from its calculations. In practice, with a finite number of optimization steps and a learning rate…
The ideas of model averaging are used to find weights in peak-over-threshold problems using a possible range of thresholds. A range of the largest observations are chosen and considered as possible thresholds, each time performing…
Stochastic gradient methods are among the most widely used algorithms for large-scale optimization and machine learning. A key technique for improving the statistical efficiency and stability of these methods is the use of averaging schemes…
The goal of this paper is an exhaustive investigation of the link between the tail measure of a regularly varying time series and its spectral tail process, independently introduced in Owada and Samorodnitsky (2012) and Basrak and Segers…
Given a finite collection of stochastic alternatives, we study the problem of sequentially allocating a fixed sampling budget to identify the optimal alternative with a high probability, where the optimal alternative is defined as the one…
At high levels, the asymptotic distribution of a stationary, regularly varying Markov chain is conveniently given by its tail process. The latter takes the form of a geometric random walk, the increment distribution depending on the sign of…
We study the finite-time behaviour of the popular temporal difference (TD) learning algorithm when combined with tail-averaging. We derive finite time bounds on the parameter error of the tail-averaged TD iterate under a step-size choice…
The tail measure of a regularly varying stationary time series has been recently introduced. It is used in this contribution to reconsider certain properties of the tail process and establish new ones. A new formulation of the time change…
Ensemble learning is a popular technique to improve the accuracy of machine learning models. It traditionally hinges on the rationale that aggregating multiple weak models can lead to better models with lower variance and hence higher…
We introduce a new type of estimator for the spectral tail process of a regularly varying time series. The approach is based on a characterizing invariance property of the spectral tail process, which is incorporated into the new estimator…
We derive the tail inequalities between two random variables starting from inequalities between its moment, or more generally between its Lebesgue-Riesz norms, which holds true on certain sets of parameters. We consider some applications…
Continual learning, an important aspect of artificial intelligence and machine learning research, focuses on developing models that learn and adapt to new tasks while retaining previously acquired knowledge. Existing continual learning…
We study the empirical version of halfspace depths with the objective of establishing a connection between the rates of convergence and the tail behaviour of the corresponding underlying distributions. The intricate interplay between the…
A common bottleneck in evaluating extremal performance measures is that, due to their very nature, tail data are often very limited. The conventional approach selects the best probability distribution from tail data using parametric…
Excellent tail performance is crucial for modern machine learning tasks, such as algorithmic fairness, class imbalance, and risk-sensitive decision making, as it ensures the effective handling of challenging samples within a dataset. Tail…
The paper presents an efficient method for simulating the tails of a target variable Z=h(X) which depends on a set of basic variables X=(X_1, ..., X_n). To this aim, variables X_i, i=1, ..., n are sequentially simulated in such a manner…
A general rate estimation method is proposed that is based on studying the in-sample evolution of appropriately chosen diverging/converging statistics. The proposed rate estimators are based on simple least squares arguments, and are shown…
Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of…
For the problem of estimating lower tail and upper tail copulas, we propose two bootstrap procedures for approximating the distribution of the corresponding empirical tail copulas. The first method uses a multiplier bootstrap of the…
While stochastic gradient descent (SGD) is one of the major workhorses in machine learning, the learning properties of many practically used variants are poorly understood. In this paper, we consider least squares learning in a…