English
Related papers

Related papers: Low probability states, data statistics, and entro…

200 papers

To quantify the complexity of a system, entropy-based methods have received considerable critical attentions in real-world data analysis. Among numerous entropy algorithms, amplitude-based formulas, represented by Sample Entropy, suffer…

Signal Processing · Electrical Eng. & Systems 2022-01-12 Hongjian Xiao , Danilo P. Mandic

Entropy estimation plays a significant role in biology, economics, physics, communication engineering and other disciplines. It is increasingly used in software engineering, e.g. in software confidentiality, software testing, predictive…

Information Theory · Computer Science 2025-01-22 Ilaria Pia la Torre , David A. Kelly , Hector D. Menendez , David Clark

The exact expression for the probability density $p_{_N}(x)$ for sums of a finite number $N$ of random independent terms is obtained. It is shown that the very tail of $p_{_N}(x)$ has a Gaussian form if and only if all the random terms are…

Probability · Mathematics 2013-05-29 Michael I. Tribelsky

We propose a Bayesian elastic net that uses empirical likelihood and develop an efficient tuning of Hamiltonian Monte Carlo for posterior sampling. The proposed model relaxes the assumptions on the identity of the error distribution,…

Methodology · Statistics 2022-07-20 Chul Moon , Adel Bedoui

In this paper, we investigate the extreme-value methodology, to propose an improved estimator of the conditional tail expectation ($CTE$) for a loss distribution with a finite mean but infinite variance. The present work introduces a new…

Statistics Theory · Mathematics 2020-02-11 Mohamed Laidi , Abdelaziz Rassoul , Hamid Ould Rouis

We show that the full-sample bootstrap is asymptotically valid for constructing confidence intervals for high-quantiles, tail probabilities, and other tail parameters of a univariate distribution. This resolves the doubts that have been…

Statistics Theory · Mathematics 2020-04-28 Svetlana Litvinova , Mervyn J. Silvapulle

In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…

Information Theory · Computer Science 2020-08-13 Mladen Kovačević , Ivan Stanojević , Vojin Šenk

Symbolic dynamics has proven to be an invaluable tool in analyzing the mechanisms that lead to unpredictability and random behavior in nonlinear dynamical systems. Surprisingly, a discrete partition of continuous state space can produce a…

Machine Learning · Computer Science 2007-07-13 Christopher C. Strelioff , James P. Crutchfield

Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…

Statistical Mechanics · Physics 2007-05-23 Roberto Luzzi , Áurea R. Vasconcellos , J. Galvão Ramos

Gibbs states are familiar from statistical mechanics, yet their use is not limited to that domain. For instance, they also feature in the maximum entropy reconstruction of quantum states from incomplete measurement data. Outside the…

Quantum Physics · Physics 2011-07-04 Jochen Rau

The problem of the insensitivity of the macroscopic behavior of any thermodynamical system to partitioning generates a bias between the reproducibility of its macroscopic behavior viewed as the simplest form of causality and its long-term…

General Physics · Physics 2007-05-23 Maria K. Koleva

In this paper we consider the estimation problem for high quantiles of a heavy-tailed distribution from block data when only a few largest values are observed within blocks. We propose estimators for high quantiles and prove that these…

Statistics Theory · Mathematics 2023-06-27 Yongcheng Qi , Mengzi Xie , Jingping Yang

We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…

Data Structures and Algorithms · Computer Science 2026-05-25 Clément L. Canonne , Yash Pote , Jonathan Scarlett , Joy Qiping Yang

Understanding how different classes are distributed in an unlabeled data set is an important challenge for the calibration of probabilistic classifiers and uncertainty quantification. Approaches like adjusted classify and count, black-box…

Machine Learning · Statistics 2024-06-19 Albert Ziegler , Paweł Czyż

We develop an efficient simulation algorithm for computing the tail probabilities of the infinite series $S = \sum_{n \geq 1} a_n X_n$ when random variables $X_n$ are heavy-tailed. As $S$ is the sum of infinitely many random variables, any…

Probability · Mathematics 2016-09-08 Henrik Hult , Sandeep Juneja , Karthyek Murthy

Bayesian estimation is increasingly popular for performing model based inference to support policymaking. These data are often collected from surveys under informative sampling designs where subject inclusion probabilities are designed to…

Methodology · Statistics 2018-07-13 Luis G. Leon-Novelo , Terrance D. Savitsky

Finite sample properties of random covariance-type matrices have been the subject of much research. In this paper we focus on the "lower tail" of such a matrix, and prove that it is subgaussian under a simple fourth moment assumption on the…

Probability · Mathematics 2013-12-11 Roberto Imbuzeiro Oliveira

Insurance data can be asymmetric with heavy tails, causing inadequate adjustments of the usually applied models. To deal with this issue, hierarchical models for collective risk with heavy-tails of the claims distributions that take also…

Applications · Statistics 2021-01-26 Pamela M. Chiroque-Solano , Fernando A. S. Moura

Tsallis' non-extensive entropy is extended to incorporate the dependence on affinities between the microstates of a system. At the core of our construction of the extended entropy ($\mathcal{H}$) is the concept of the effective number of…

Quantitative Methods · Quantitative Biology 2022-02-08 Keisuke Okamura

A wide range of natural and social phenomena result in observables whose distributions can be well approximated by a power-law decay. The well-known Hill estimator of the tail exponent provides results which are in many respects superior to…

Data Analysis, Statistics and Probability · Physics 2009-05-20 Éva Rácz , János Kertész , Zoltán Eisler