Related papers: Semiparametric Exponential Families for Heavy-Tail…
The Expectation Maximization (EM) algorithm is widely used as an iterative modification to maximum likelihood estimation when the data is incomplete. We focus on a semi-supervised case to learn the model from labeled and unlabeled samples.…
Standard methods for determining the number of factors often overestimate the true number when data exhibit heavy-tailed randomness, misinterpreting noise-induced outliers as genuine factors. This paper addresses this challenge within the…
For extreme value estimation we propose to use a model with a Dirichlet process mixture of gamma densities in the center and generalized Pareto densities for the tails. Due to the randomness in the center and a heavy tailed density in the…
We study an unconventional chiral random matrix model with a heavy-tailed probabilistic weight. The model is shown to exhibit chiral symmetry breaking with no bilinear condensate, in analogy to the Stern phase of QCD. We solve the model…
We consider the estimation of densities in multiple subpopulations, where the available sample size in each subpopulation greatly varies. This problem occurs in epidemiology, for example, where different diseases may share similar…
This paper addresses the problem of estimating the tail index of distributions with heavy, Pareto-type tails for dependent data, that is of interest in the areas of finance, insurance, environmental monitoring and teletraffic analysis. A…
The non-asymptotic tail bounds of random variables play crucial roles in probability, statistics, and machine learning. Despite much success in developing upper bounds on tail probability in literature, the lower bounds on tail…
The rate of uniform convergence in extreme value statistics is non-universal and can be arbitrarily slow. Further, the relative error can be unbounded in the tail of the approximation, leading to difficulty in extrapolating the extreme…
Empirical best prediction (EBP) is a well-known method for producing reliable proportion estimates when the primary data source provides only small or no sample from finite populations. There are potential challenges in implementing…
Modeling heterogeneity on heavy-tailed distributions under a regression framework is challenging, and classical statistical methodologies usually place conditions on the distribution models to facilitate the learning procedure. However,…
The paper suggests a simple method of deriving minimax lower bounds to the accuracy of statistical inference on heavy tails. A well-known result by Hall and Welsh (Ann. Statist. 12 (1984) 1079-1084) states that if $\hat{\alpha}_n$ is an…
In this paper, we propose a reduced-bias estimator of the EVI for Pareto-type tails (heavy-tailed) distributions. This is derived using the weighted least squares method. It is shown that the estimator is unbiased, consistent and…
A fundamental problem in analysis of complex systems is getting a reliable estimate of entropy of their probability distributions over the state space. This is difficult because unsampled states can contribute substantially to the entropy,…
This paper explores the applications of the 20/60/20 rule-a heuristic method that segments data into top-performing, average-performing, and underperforming groups-in mathematical finance. We review the statistical foundations of this rule…
We consider removing lower order statistics from the classical Hill estimator in extreme value statistics, and compensating for it by rescaling the remaining terms. Trajectories of these trimmed statistics as a function of the extent of…
We prove tail estimates for variables $\sum_i f(X_i)$, where $(X_i)_i$ is the trajectory of a random walk on an undirected graph (or, equivalently, a reversible Markov chain). The estimates are in terms of the maximum of the function $f$,…
The best arm identification problem requires identifying the best alternative (i.e., arm) in active experimentation using the smallest number of experiments (i.e., arm pulls), which is crucial for cost-efficient and timely decision-making…
Heavy tailed phenomena are naturally analyzed by extreme value statistics. A crucial step in such an analysis is the estimation of the extreme value index, which describes the tail heaviness of the underlying probability distribution. We…
Heavy-tailed distributions, prevalent in a lot of real-world applications such as finance, telecommunications, queuing theory, and natural language processing, are challenging to model accurately owing to their slow tail decay. Bernstein…
We study the fundamental task of outlier-robust mean estimation for heavy-tailed distributions in the presence of sparsity. Specifically, given a small number of corrupted samples from a high-dimensional heavy-tailed distribution whose mean…