Related papers: The maximum entropy of a metric space
We normalize the combinatorial Laplacian of a graph by the degree sum, look at its eigenvalues as a probability distribution and then study its Shannon entropy. Equivalently, we represent a graph with a quantum mechanical state and study…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
Abstract: The number of points $x=(x_1 ,x_2 ,...x_n)$ that lie in an integer cube $C$ in $R^n$ and satisfy the constraints $\sum_j h_{ij}(x_j )=s_i ,1\le i\le d$ is approximated by an Edgeworth-corrected Gaussian formula based on the…
This paper modifies Jaynes's axioms of plausible reasoning and derives the minimum relative entropy principle, Bayes's rule, as well as maximum likelihood from first principles. The new axioms, which I call the Optimum Information…
We revisit the classical problem of inverting dimension-reducing linear mappings using the maximum entropy (MaxEnt) criterion. In the literature, solutions are problem-dependent, inconsistent, and use different entropy measures. We propose…
Bayes' theorem incorporates distinct types of information through the likelihood and prior. Direct observations of state variables enter the likelihood and modify posterior probabilities through consistent updating. Information in terms of…
It is supposed that the exponential multiplier in the method of the non-equilibrium statistical operator (Zubarev`s approach) can be considered as a distribution density of the past lifetime of the system, and can be replaced by an…
Given a knowledge base KB containing first-order and statistical facts, we consider a principled method, called the random-worlds method, for computing a degree of belief that some formula Phi holds given KB. If we are reasoning about a…
We propose a unifying picture where the notion of generalized entropy is related to information theory by means of a group-theoretical approach. The group structure comes from the requirement that an entropy be well defined with respect to…
We study the notion of approximate entropy within the framework of network theory. Approximate entropy is an uncertainty measure originally proposed in the context of dynamical systems and time series. We firstly define a purely structural…
The Shannon entropy of a random variable has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different…
We discuss the dependence of the Shannon entropy of normalized finite rank-1 POVMs on the choice of the input state, looking for the states that minimize this quantity. To distinguish the class of measurements where the problem can be…
We study the continuity of an abstract generalization of the maximum-entropy inference - a maximizer. It is defined as a right-inverse of a linear map restricted to a convex body which uniquely maximizes on each fiber of the linear map a…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
Given a factor code $\pi$ from a one-dimensional shift of finite type $X$ onto an irreducible sofic shift $Y$, if $\pi$ is finite-to-one there is an invariant called the degree of $\pi$ which is defined the number of preimages of a typical…
The entropy of a finite probability space $X$ measures the observable cardinality of large independent products $X^{\otimes n}$ of the probability space. If two probability spaces $X$ and $Y$ have the same entropy, there is an almost…
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of…
In this paper, we present a new multi-scale information content calculation method based on Shannon information (and Shannon entropy). The original method described by Claude E. Shannon and based on the logarithm of the probability of…