Related papers: On a minimum distance procedure for threshold sele…
We propose an approach to estimate the number of samples required for a model to reach a target performance. We find that the power law, the de facto principle to estimate model performance, leads to large error when using a small dataset…
In order to probe with high precision the tails of the ground-state energy distribution of disordered spin systems, K\"orner, Katzgraber and Hartmann \cite{Ko_Ka_Ha} have recently proposed an importance-sampling Monte-Carlo Markov chain in…
This article presents a statistical analysis method and introduces the corresponding software package "tailstat," which is believed to be widely applicable to today's internet society. The proposed method facilitates statistical analyses…
We consider the estimation of small probabilities or other risk quantities associated with rare but catastrophic events. In the model-based literature, much of the focus has been devoted to efficient Monte Carlo computation or analytical…
We fit the exponent of the Pareto distribution, that is equivalent or can approximate the continuous power law distribution given a cutoff point, using linear regression (LR). We use LR on the logged variables of the empirical tail (one…
We study upper and lower bounds on the sample-complexity of learning near-optimal behaviour in finite-state discounted Markov Decision Processes (MDPs). For the upper bound we make the assumption that each action leads to at most two…
Statistical distributions with heavy tails are ubiquitous in natural and social phenomena. Since the entries in heavy tail have disproportional significance, the knowledge of its exact shape is very important. Citations of scientific papers…
Accurately quantifying tail risks-rare but high-impact events such as financial crashes or extreme weather-is a central challenge in risk management, with serially dependent data. We develop a Bayesian framework based on the Generalized…
We introduce the Thresholding Monte Carlo Tree Search problem, in which, given a tree $\mathcal{T}$ and a threshold $\theta$, a player must answer whether the root node value of $\mathcal{T}$ is at least $\theta$ or not. In the given tree,…
We consider random-design linear prediction and related questions on the lower tail of random matrices. It is known that, under boundedness constraints, the minimax risk is of order $d/n$ in dimension $d$ with $n$ samples. Here, we study…
Language models are increasingly capable and are being rapidly deployed on a population-level scale. As a result, the safety of these models is increasingly high-stakes. Fortunately, advances in alignment have significantly reduced the…
We examine the performance of six estimators of the power-law cross-correlations -- the detrended cross-correlation analysis, the detrending moving-average cross-correlation analysis, the height cross-correlation analysis, the averaged…
We suggest a robust nearest-neighbor approach to classifying high-dimensional data. The method enhances sensitivity by employing a threshold and truncates to a sequence of zeros and ones in order to reduce the deleterious impact of…
The subject of tail estimation for randomly censored data from a heavy tailed distribution receives growing attention, motivated by applications for instance in actuarial statistics. The bias of the available estimators of the extreme value…
In a number of applications, particularly in financial and actuarial mathematics, it is of interest to characterize the tail distribution of a random variable $V$ satisfying the distributional equation $V\stackrel{\mathcal{D}}{=}f(V)$,…
In the last fifteen the subset sampling method has often been used in reliability problems as a tool for calculating small probabilities. This method is extrapolating from an initial Monte Carlo estimate for the probability content of a…
This short communication uses a simple experiment to show that fitting to a power law distribution by using graphical methods based on linear fit on the log-log scale is biased and inaccurate. It shows that using maximum likelihood…
In this work, we provide robust bounds on the tail probabilities and the tail index of heavy-tailed distributions in the context of model misspecification. They are defined as the optimal value when computing the worst-case tail behavior…
To consider a high-dimensional random process, we propose a notion about stochastic tensor-valued random process (TRP). In this work, we first attempt to apply a generic chaining method to derive tail bounds for all p-th moments of the…
The Hill estimator is often used to infer the power behavior in tails of experimental distribution functions. This estimator is known to produce bad results in certain situations which have lead to the so-called Hill horror plots. In this…