Related papers: Maximum Entropy and the Glasses You Are Looking Th…
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's…
We take another look at the general problem of selecting a preferred probability measure among those that comply with some given constraints. The dominant role that entropy maximization has obtained in this context is questioned by arguing…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…
The Principle of Maximum Entropy, a powerful and general method for inferring the distribution function given a set of constraints, is applied to deduce the overall distribution of 3D plasmoids (flux ropes/tubes) for systems where resistive…
We study maximum-entropy inference for finite-dimensional quantum states under linear moment constraints. Given expectation values of finitely many observables, the feasible set of states is convex but typically non-unique. The…
Entropy is a very useful concept from physics that tries to explain how a system behaves from a point of view of the thermodynamics. However, there are two ways to explain entropy, and it depends on if we are studying a microsystem or a…
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…
Recent theoretical results renewed the interest in charged particle multiplicity distributions. The Shannon entropy of such distributions is conjectured to be related to the entanglement or von Neumann entropy of partonic quantum system. In…
We consider zero-sum repeated games in which the players are restricted to strategies that require only a limited amount of randomness. Let $v_n$ be the max-min value of the $n$ stage game; previous works have characterized…
Maximal inequalities refer to bounds on expected values of the supremum of averages of random variables over a collection. They play a crucial role in the study of non-parametric and high-dimensional estimators, and especially in the study…
Sanov's Theorem and the Conditional Limit Theorem (CoLT) are established for a multicolor Polya Eggenberger urn sampling scheme, giving the Polya divergence and the Polya extension to the Maximum Relative Entropy (MaxEnt) method. Polya…
We study minimum entropy submodular optimization, a common generalization of the minimum entropy set cover problem, studied earlier by Cardinal et al., and the submodular set cover problem. We give a general bound of the approximation…
Using results from neurobiology on perceptual decision making and value-based decision making, the problem of decision making between lotteries is reformulated in an abstract space where uncertain prospects are mapped to corresponding…
Maximum likelihood estimation is a valuable tool often applied to inverse problems in quantum theory. Estimation from small data sets can, however, have non unique solutions. We discuss this problem and propose to use Jaynes maximum entropy…
Recent work in data mining and related areas has highlighted the importance of the statistical assessment of data mining results. Crucial to this endeavour is the choice of a non-trivial null model for the data, to which the found patterns…
We propose a game-theoretic framework that incorporates both incomplete information and general ambiguity attitudes on factors external to all players. Our starting point is players' preferences on payoff-distribution vectors, essentially…
We provide an elementary proof that revenue-maximizing mechanisms exist in multi-parameter settings whenever the distribution of valuations has finite expectation.
The Maximum Entropy Method (MEM) is a popular data analysis technique based on Bayesian inference, which has found various applications in the research literature. While the MEM itself is well-grounded in statistics, I argue that its…
The diversity of a community that cannot be fully counted must be inferred. The two preeminent inference methods are the MaxEnt method, which uses information in the form of constraints and Bayes' rule which uses information in the form of…
The relationship between three probability distributions and their maximizable entropy forms is discussed without postulating entropy property. For this purpose, the entropy I is defined as a measure of uncertainty of the probability…