Related papers: An information-theoretic upper bound on prime gaps
In the framework of Cramer's probabilistic model of primes, we explore the exact and asymptotic distributions of maximal prime gaps. We show that the Gumbel extreme value distribution exp(-exp(-x)) is the limit law for maximal gaps between…
Maximum entropy principle (MEP) offers an effective and unbiased approach to inferring unknown probability distributions when faced with incomplete information, while neural networks provide the flexibility to learn complex distributions…
The Cross Entropy method is a well-known adaptive importance sampling method for rare-event probability estimation, which requires estimating an optimal importance sampling density within a parametric class. In this article we estimate an…
This note is concerned with weakly interacting stochastic particle systems with possibly singular pairwise interactions. In this setting, we observe a connection between entropic propagation of chaos and exponential concentration bounds for…
Though neural network models demonstrate impressive performance, we do not understand exactly how these black-box models make individual predictions. This drawback has led to substantial research devoted to understand these models in areas…
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
The Adaptive Multilevel Splitting algorithm is a very powerful and versatile iterative method to estimate the probability of rare events, based on an interacting particle systems. In an other article, in a so-called idealized setting, the…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
Lower bounds for the R\'enyi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in…
The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
Rooted trees with probabilities are used to analyze properties of a variable length code. A bound is derived on the difference between the entropy rates of the code and a memoryless source. The bound is in terms of normalized informational…
We explore a method of statistical estimation called Maximum Entropy on the Mean (MEM) which is based on an information-driven criterion that quantifies the compliance of a given point with a reference prior probability measure. At the core…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
Detecting rare events, those defined to give rise to high impact but have a low probability of occurring, is a challenge in a number of domains including meteorological, environmental, financial and economic. The use of machine learning to…
An information-theoretic framework is introduced to analyze last-layer embedding, focusing on learned representations for regression tasks. We define representation-rate and derive limits on the reliability with which input-output…
A random set is a generalisation of a random variable, i.e. a set-valued random variable. The random set theory allows a unification of other uncertainty descriptions such as interval variable, mass belief function in Dempster-Shafer theory…
Finding upper limits on the rate of events from a proposed process in the presence of unknown backgrounds is an often encountered problem in the search for rare processes. Methods based on unusually large "gaps", or spacings, in the event…
Many probabilistic inference problems such as stochastic filtering or the computation of rare event probabilities require model analysis under initial and terminal constraints. We propose a solution to this bridging problem for the widely…