Related papers: Maximum entropy and the problem of moments: A stab…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
We establish a set of equations for moments of the distribution function. In the relaxation time approximations, these moments obey a coupled set of equations that can be truncated order-by-order. Solving the equations of moments, we are…
The montecarlo method, which is quite commonly used to solve maximum entropy problems in statistical physics, can actually be used to solve inverse problems in a much wider context. The probability distribution which maximizes entropy can…
We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…
Stein operators allow to characterise probability distributions via differential operators. Based on these characterisations, we develop a new method of point estimation for marginal parameters of strictly stationary and ergodic processes,…
We study maximum-entropy inference for finite-dimensional quantum states under linear moment constraints. Given expectation values of finitely many observables, the feasible set of states is convex but typically non-unique. The…
Compressed Counting (CC) [22] was recently proposed for estimating the ath frequency moments of data streams, where 0 < a <= 2. CC can be used for estimating Shannon entropy, which can be approximated by certain functions of the ath…
The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that…
A moment body is a linear projection of the spectraplex, the convex set of trace-one positive semidefinite matrices. Determining whether a given point lies within a given moment body is a problem with numerous applications in quantum state…
We study the problem of computing the \textsc{Maxima} of a set of $n$ $d$-dimensional points. For dimensions 2 and 3, there are algorithms to solve the problem with order-oblivious instance-optimal running time. However, in higher…
The maximal entropy moment method (MEM) is systematic solution of the challenging problem: generating extended hydrodynamic equations valid for both dense and rarefied gases. However, simulating MEM suffers from a computational expensive…
Particle distributions in weakly collisional environments such as the magnetosphere have been observed to show deviations from the Maxwellian distribution. These can often be reproduced in kinetic simulations, but fluid models, which are…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
Optimization results are one method for understanding neural computation from Nature's perspective and for defining the physical limits on neuron-like engineering. Earlier work looks at individual properties or performance criteria and…
We present a study of the equilibration process of nonequilibrium systems by means of molecular dynamics simulation technique. The nonequilibrium conditions are achieved in systems by defining velocity components of the constituent atoms…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…
The selection of an equilibrium state by maximising the entropy of a system, subject to certain constraints, is often powerfully motivated as an exercise in logical inference, a procedure where conclusions are reached on the basis of…
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…
We develop the method of Maximum Entropy (ME) as a technique to generate approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…