Related papers: Heavy-Tailed NGG Mixture Models
In the study of heavy tail data, several models have been introduced. If the interest is in the tail of the distribution, block maxima or excess over thresholds are the typical approaches, wasting relevant information in the bulk of the…
An important question in health services research is the estimation of the proportion of medical expenditures that exceed a given threshold. Typically, medical expenditures present highly skewed, heavy tailed distributions, for which (a)…
Despite the successes of probabilistic models based on passing noise through neural networks, recent work has identified that such methods often fail to capture tail behavior accurately, unless the tails of the base distribution are…
Unraveling the reasons behind the remarkable success and exceptional generalization capabilities of deep neural networks presents a formidable challenge. Recent insights from random matrix theory, specifically those concerning the spectral…
It has repeatedly been observed that loss minimization by stochastic gradient descent (SGD) leads to heavy-tailed distributions of neural network parameters. Here, we analyze a continuous diffusion approximation of SGD, called homogenized…
Recent theoretical and empirical successes in deep learning, including the celebrated neural scaling laws, are punctuated by the observation that many objects of interest tend to exhibit some form of heavy-tailed or power law behavior. In…
Continuous mixtures of distributions are widely employed in the statistical literature as models for phenomena with highly divergent outcomes; in particular, many familiar heavy-tailed distributions arise naturally as mixtures of…
We discuss non-Gaussian random matrices whose elements are random variables with heavy-tailed probability distributions. In probability theory heavy tails of the distributions describe rare but violent events which usually have dominant…
Heavy-tailed distributions are found throughout many naturally occurring phenomena. We have reviewed the models of stochastic dynamics that lead to heavy-tailed distributions (and power law distributions, in particular) including the…
Normalizing flows are a flexible class of probability distributions, expressed as transformations of a simple base distribution. A limitation of standard normalizing flows is representing distributions with heavy tails, which arise in…
For extreme value estimation we propose to use a model with a Dirichlet process mixture of gamma densities in the center and generalized Pareto densities for the tails. Due to the randomness in the center and a heavy tailed density in the…
The family of location and scale mixtures of Gaussians has the ability to generate a number of flexible distributional forms. It nests as particular cases several important asymmetric distributions like the Generalised Hyperbolic…
Modelling non-homogeneous and multi-component data is a problem that challenges scientific researchers in several fields. In general, it is not possible to find a simple and closed form probabilistic model to describe such data. That is why…
Recent empirical evidence indicates that many machine learning applications involve heavy-tailed gradient noise, which challenges the standard assumptions of bounded variance in stochastic optimization. Gradient clipping has emerged as a…
The dominant approaches to text representation in natural language rely on learning embeddings on massive corpora which have convenient properties such as compositionality and distance preservation. In this paper, we develop a novel method…
Recent theoretical studies have shown that heavy-tails can emerge in stochastic optimization due to `multiplicative noise', even under surprisingly simple settings, such as linear regression with Gaussian data. While these studies have…
We characterise the learning of a mixture of two clouds of data points with generic centroids via empirical risk minimisation in the high dimensional regime, under the assumptions of generic convex loss and convex regularisation. Each cloud…
The presence of non-Gaussian tails is a prevalent characteristic in many financial modeling scenarios, necessitating the use of complex non-Gaussian distributions such as the generalized beta of the second kind (GB2) and the skewed…
Heavy-tailed probability distributions are extremely useful and play a crucial role in modeling different types of financial data sets. This study presents a two-pronged methodology. First, a mixture probability distribution is created by…
Recent works have proposed incorporating heavy-tailed (HT) noise into diffusion- and flow-based generative models, with the goals of better recovering the tails of target distributions and improving generative diversity. This motivation is…