Related papers: Hierarchical Maximum Entropy via the Renormalizati…
We generalize the usual exponential Boltzmann factor to any reasonable and potentially observable distribution function, $B(E)$. By defining generalized logarithms $\Lambda$ as inverses of these distribution functions, we are led to a…
Using statistical thermodynamics, we derive a general expression of the stationary probability distribution for thermodynamic systems driven out of equilibrium by several thermodynamic forces. The local equilibrium is defined by imposing…
New experimental methods make it possible to measure the expression levels of many genes, simultaneously, in snapshots from thousands or even millions of individual cells. Current approaches to analyze these experiments involve clustering…
We propose an additional category of dimensionless groups based on the principle of {\it entropic similarity}, defined by ratios of (i) entropy production terms; (ii) entropy flow rates or fluxes; or (iii) information flow rates or fluxes.…
With the tremendous success of deep learning in visual tasks, the representations extracted from intermediate layers of learned models, that is, deep features, attract much attention of researchers. Previous empirical analysis shows that…
We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…
The structure of interactions in most of animals and human societies can be best represented by complex hierarchical networks. In order to maintain close to optimal functioning both stability and adaptability are necessary. Here we…
Motivated by bit threads, we introduce a new prescription for computing entropy vectors outside the holographic entropy cone. By utilizing cycle flows on directed graphs, we show that the maximum cycle flow associated to any subset of…
Spiking activity from populations of neurons display causal interactions and memory effects. Therefore, they are expected to show some degree of irreversibility in time. Motivated by the spike train statistics, in this paper we build a…
Ensembles of networks arise in many scientific fields, but there are few statistical tools for inferring their generative processes, particularly in the presence of both dyadic dependence and cross-graph heterogeneity. To fill in this gap,…
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased"…
We discuss basic statistical properties of systems with multifractal structure. This is possible by extending the notion of the usual Gibbs--Shannon entropy into more general framework - Renyi's information entropy. We address the…
Hierarchical structures are ubiquitous in human and animal societies, but a fundamental understanding of their raison d'\^etre has been lacking. Here, we present a general theory in which hierarchies are obtained as the optimal design that…
Hierarchies are of fundamental interest in both stochastic optimal control and biological control due to their facilitation of a range of desirable computational traits in a control algorithm and the possibility that they may form a core…
The Boltzmann entropy $S^{(B)}$ is true in the case of equal probability of all microstates of a system. In the opposite case it should be averaged over all microstates that gives rise to the Boltzmann--Shannon entropy (BSE). Maximum…
Using a maximum entropy principle to assign a statistical weight to any graph, we introduce a model of random graphs with arbitrary degree distribution in the framework of standard statistical mechanics. We compute the free energy and the…
In the previous papers (Kui\'{c} et al. in Found Phys 42:319-339, 2012; Kui\'{c} in arXiv:1506.02622, 2015), it was demonstrated that applying the principle of maximum information entropy by maximizing the conditional information entropy,…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
We derive entropy factorization estimates for spin systems using the stochastic localization approach proposed by Eldan and Chen-Eldan, which, in this context, is equivalent to the renormalization group approach developed independently by…