Related papers: Hard Core entropy: lower bounds
When constructing models of the world, we aim for optimal compressions: models that include as few details as possible while remaining as accurate as possible. But which details -- or features measured in data -- should we choose to include…
Let $\mathcal{H}$ be a hypergraph with $n$ vertices. Suppose that $d_1,d_2,\ldots,d_n$ are degrees of the vertices of $\mathcal{H}$. The $t$-th graph entropy based on degrees of $\mathcal{H}$ is defined as $$ I_d^t(\mathcal{H})…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
A general method is presented for modeling high entropy alloys as ensembles of randomly sampled, ordered configurations on a given lattice. Statistical mechanics is applied post hoc to derive the ensemble properties as a function of…
Any coded subshift X defined by a set C of code words contains a subshift, which we call L, consisting of limits of single code words. We show that when C satisfies a unique decomposition property, the topological entropy h(X) of X is…
In many high-impact applications, it is important to ensure the quality of output of a machine learning algorithm as well as its reliability in comparison with the complexity of the algorithm used. In this paper, we have initiated a…
Invariance entropy is a measure for the smallest data rate in a noiseless digital channel above which a controller that only receives state information through this channel is able to render a given subset of the state space invariant. In…
Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic…
The configurational entropy of several H-disordered ice polymorphs is calculated by means of a thermodynamic integration along a path between a totally H-disordered state and one fulfilling the Bernal-Fowler ice rules. A Monte Carlo…
Motivated by the approach of random linear codes, a new distance in the vector space over a finite field is defined as the logarithm of the "surface area" of a Hamming ball with radius being the corresponding Hamming distance. It is named…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
We provide another approach to Friedland's result that the topological entropy $h$ of a symmetric nearest-neighbor subshift is computable. Instead of the previous algebraic technique, our approach is mostly combinatorial and involves only…
Living systems maintain or increase local order by working against the Second Law of Thermodynamics. Thermodynamic consistency is restored as they dissipate heat, thereby increasing the net entropy of their environment. Recently introduced…
We define and investigate a notion of entropy for quantum error correcting codes. The entropy of a code for a given quantum channel has a number of equivalent realisations, such as through the coefficients associated with the Knill-Laflamme…
We give an upper bound for the topological entropy of maps on inverse limit spaces in terms of their set-valued components. In a special case of a diagonal map on the inverse limit space $\underleftarrow{\lim}(I,f)$, where every diagonal…
We consider holographic entanglement entropy in AdS black hole backgrounds by using the limit of large number of dimensions. By dividing the geometry to two patches (with one patch covering the vicinity of the black hole horizon and another…
A maximum entropy-based framework is presented for the synthesis of projections from multiple Earth climate models. This identifies the most representative (most probable) model from a set of climate models -- as defined by specified…
Proteins are essential components of living systems, capable of performing a huge variety of tasks at the molecular level, such as recognition, signalling, copy, transport, ... The protein sequences realizing a given function may largely…
We derive an upper bound on the maximum balanced bipartite entanglement entropy of ground states of many-body Hamiltonians defined on a graph, agnostic to any particular model, that possesses a nontrivial automorphism group. We show that…