Related papers: Entropy Concentration and the Empirical Coding Gam…
The possibility of reconciliation between canonical probability distributions obtained from the $q$-maximum entropy principle with predictions from the law of large numbers when empirical samples are held to the same constraints, is…
Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…
We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the…
We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the…
We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
Generalized entropies and relative entropies are the subject of active research. Similar to the standard relative entropy, the relative $q$-entropy is generally unbounded for $q>1$. Upper bounds on the quantum relative $q$-entropy in terms…
The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…
This work has two contributions. The first one is extending the Large Deviation Principle for uniform hyper-graphons from Lubetzky and Zhao \cite{lubetzky2015replica} to the multi-relational setting where each hyper-graphon can have…
The Principle of Insufficient Reason (PIR) assigns equal probabilities to each alternative of a random experiment whenever there is no reason to prefer one over the other. The Maximum Entropy Principle (MaxEnt) generalizes PIR to the case…
A statistical model for the fragmentation of a conserved quantity is analyzed, using the principle of maximum entropy and the theory of partitions. Upper and lower bounds for the restricted partitioning problem are derived and applied to…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
Entropy is a concept that has traditionally been reliant on a definite notion of causality. However, without a definite notion of causality, the concept of entropy is not all lost. Indefinite causal structure results from combining…
Exponential random graph models have attracted significant research attention over the past decades. These models are maximum-entropy ensembles under the constraints that the expected values of a set of graph observables are equal to given…
We present a general holistic theory for the organization of complex networks, both human-engineered and naturally-evolved. Introducing concepts of value of interactions and satisfaction as generic network performance measures, we show that…
A theory to predict the steady state position of a dissipative, flow-controlled system, as defined by a control volume, is developed based on the Maximum Entropy (MaxEnt) principle of Jaynes, involving minimisation of a generalised free…
Language prediction is constrained by informational entropy intrinsic to language, such that there exists a limit to how accurate any language model can become and equivalently a lower bound to language compression. The most efficient…
Comparison-based algorithms are algorithms for which the execution of each operation is solely based on the outcome of a series of comparisons between elements. Comparison-based computations can be naturally represented via the following…
We propose an approach to the realization of many-body quantum state distributions inspired by combined principles of thermodynamics and mesoscopic physics. Its essence is a maximum entropy principle conditioned by conservation laws. We go…
A novel definition of the conditional smooth Renyi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Renyi entropy is appropriate to give lower and upper…