English
Related papers

Related papers: On Optimization over Tail Distributions

200 papers

Stochastic optimization problems often involve data distributions that change in reaction to the decision variables. This is the case for example when members of the population respond to a deployed classifier by manipulating their features…

Optimization and Control · Mathematics 2020-12-15 Dmitriy Drusvyatskiy , Lin Xiao

The extreme value dependence of regularly varying stationary time series can be described by the spectral tail process. Drees, Segers and Warchol [Extremes 18(3): 369--402, 2015] proposed estimators of the marginal distributions of this…

Statistics Theory · Mathematics 2019-07-23 Holger Drees , Miran Knezevic

This paper presents a novel semiparametric method to study the effects of extreme events on binary outcomes and subsequently forecast future outcomes. Our approach, based on Bayes' theorem and regularly varying (RV) functions, facilitates a…

Econometrics · Economics 2025-02-25 Laura Liu , Yulong Wang

We investigate the relaxation of long-tailed distributions under stochastic dynamics that do not support such tails. Linear relaxation is found to be a borderline case in which long tails are exponentially suppressed in time but not…

Statistical Mechanics · Physics 2015-06-22 Christian Van den Broeck , Upendra Harbola , Raul Toral , Katja Lindenberg

Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error. While these studies have shed light on interesting aspects of the generalization…

Machine Learning · Statistics 2023-02-14 Anant Raj , Melih Barsbey , Mert Gürbüzbalaban , Lingjiong Zhu , Umut Şimşekli

This paper investigates the use of extreme value theory for modelling the distribution of demand-net-of-wind for capacity adequacy assessment. Extreme value theory approaches are well-established and mathematically justified methods for…

Applications · Statistics 2019-07-31 Amy L Wilson , Stan Zachary

Extreme quantile treatment effects (eQTEs) measure the causal impact of a treatment on the tails of an outcome distribution and are central for studying rare, high-impact events. Standard QTE methods often fail in extreme regimes due to…

Methodology · Statistics 2026-03-25 Mengran Li , Daniela Castro-Camilo

Many stochastic optimization problems include chance constraints that enforce constraint satisfaction with a specific probability; however, solving an optimization problem with chance constraints assumes that the solver has access to the…

Optimization and Control · Mathematics 2021-09-21 Joshua Comden , Ahmed S. Zamzam , Andrey Bernstein

Extremile regression, as a least squares analog of quantile regression, is potentially useful tool for modeling and understanding the extreme tails of a distribution. However, existing extremile regression methods, as nonparametric…

Methodology · Statistics 2025-07-03 Rong Jiang , Keming Yu , Jiangfeng Wang

We consider removing lower order statistics from the classical Hill estimator in extreme value statistics, and compensating for it by rescaling the remaining terms. Trajectories of these trimmed statistics as a function of the extent of…

Methodology · Statistics 2020-06-30 Martin Bladt , Hansjoerg Albrecher , Jan Beirlant

Accurate goodness-of-fit tests for the extreme tails of empirical distributions is a very important issue, relevant in many contexts, including geophysics, insurance, and finance. We have derived exact asymptotic results for a…

Applications · Statistics 2012-10-12 Rémy Chicheportiche , Jean-Philippe Bouchaud

Modern statistical analyses often encounter datasets with massive sizes and heavy-tailed distributions. For datasets with massive sizes, traditional estimation methods can hardly be used to estimate the extreme value index directly. To…

Methodology · Statistics 2022-07-26 Yongxin Li , Liujun Chen , Deyuan Li , Hansheng Wang

This paper attempts to provide a decision-theoretic foundation for the measurement of economic tail risk, which is not only closely related to utility theory but also relevant to statistical model uncertainty. The main result is that the…

Risk Management · Quantitative Finance 2015-08-18 Steven Kou , Xianhua Peng

Models for extreme values are generally derived from limit results, which are meant to be good enough approximations when applied to finite samples. Depending on the speed of convergence of the process underlying the data, these…

Statistics Theory · Mathematics 2019-02-20 Thomas Lugrin , Anthony C. Davison , Jonathan A. Tawn

The aim of this paper is to provide several novel upper bounds on the excess risk with a primal focus on classification problems. We suggest two approaches and the obtained bounds are represented via the distribution dependent local…

Statistics Theory · Mathematics 2018-03-13 Nikita Zhivotovskiy

Understanding the generalization properties of heavy-tailed stochastic optimization algorithms has attracted increasing attention over the past years. While illuminating interesting aspects of stochastic optimizers by using heavy-tailed…

Machine Learning · Statistics 2025-06-03 Benjamin Dupuis , Umut Şimşekli

Existing theory for multivariate extreme values focuses upon characterizations of the distributional tails when all components of a random vector, standardized to identical margins, grow at the same rate. In this paper, we consider the…

Statistics Theory · Mathematics 2013-12-20 J. L. Wadsworth , J. A. Tawn

In the world of multivariate extremes, estimation of the dependence structure still presents a challenge and an interesting problem. A procedure for the bivariate case is presented that opens the road to a similar way of handling the…

Statistics Theory · Mathematics 2008-11-14 John H. J. Einmahl , Andrea Krajina , Johan Segers

We study the problem of factor modelling vector- and tensor-valued time series in the presence of heavy tails in the data, which produce extreme observations with non-negligible probability. We propose to combine a two-step procedure for…

Methodology · Statistics 2025-09-08 Matteo Barigozzi , Haeran Cho , Hyeyoung Maeng

Model selection is often performed by empirical risk minimization. The quality of selection in a given situation can be assessed by risk bounds, which require assumptions both on the margin and the tails of the losses used. Starting with…

Statistics Theory · Mathematics 2008-12-18 Charles Mitchell , Sara van de Geer