Related papers: Entropy Bounds for Discrete Random Variables via M…
We develop the method of Maximum Entropy (ME) as a technique to generate approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
The method of maximum entropy is quite a powerful tool to solve the generalized moment problem, which consists of determining the probability density of a random variable X from the knowledge of the expected values of a few functions of the…
In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data…
In this letter, we give a concise, closed-form expression for the differential entropy of the sum of two independent, non-identically-distributed exponential random variables. The derivation is straightforward, but such a concise entropy…
For the basic maximum likelihood estimating function of the two parameters Weibull distribution, a simple proof on its global monotonicity is given to ensure the existence and uniqueness of its solution. The boundary of the function's…
The Random Permutation Set (RPS) is a new type of set proposed recently, which can be regarded as the generalization of evidence theory. To measure the uncertainty of RPS, the entropy of RPS and its corresponding maximum entropy have been…
We suggest approximating the distribution of the sum of independent and identically distributed random variables with a Pareto-like tail by combining extreme value approximations for the largest summands with a normal approximation for the…
We deal with stochastic differential equations with jumps. In order to obtain an accurate approximation scheme, it is usual to replace the "small jumps" by a Brownian motion. In this paper, we prove that for every fixed time $t$, the…
The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…
In this paper relations among some kinds of cumulative entropies and moments of order statistics are presented. By using some characterizations and the symmetry of a non negative and absolutely continuous random variable X, lower and upper…
We consider upper exponential bounds for the probability of the event that an absolute deviation of sample mean from mathematical expectation p is bigger comparing with some ordered level epsilon. These bounds include 2 coefficients {alpha,…
An often-cited fact regarding mixing or mixture distributions is that their density functions are able to approximate the density function of any unknown distribution to arbitrary degrees of accuracy, provided that the mixing or mixture…
The field of complex networks studies a wide variety of interacting systems by representing them as networks. To understand their properties and mutual relations, the randomisation of network connections is a commonly used tool. However,…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
Solutions to conservation laws satisfy the monotonicity property: the number of local extrema is a non-increasing function of time, and local maximum/minimum values decrease/increase monotonically in time. This paper investigates this…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
Maximum entropy method is a constructive criterion for setting up a probability distribution maximally non-committal to missing information on the basis of partial knowledge, usually stated as constrains on expectation values of some…
This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
In [Schuhmacher, Electron. J. Probab. 10 (2005), 165--201] estimates of the Barbour-Brown distance d_2 between the distribution of a thinned point process and the distribution of a Poisson process were derived by combining discretization…