Related papers: Minimum and maximum entropy distributions for bina…
We consider an extension of the conditional min- and max-entropies to infinite-dimensional separable Hilbert spaces. We show that these satisfy characterizing properties known from the finite-dimensional case, and retain…
Recent literature in the last Maximum Entropy workshop introduced an analogy between cumulative probability distributions and normalized utility functions. Based on this analogy, a utility density function can de defined as the derivative…
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example.…
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator…
Pairwise maximum-entropy models have been used in recent neuroscientific literature to predict the activity of neuronal populations, given only the time-averaged correlations of the neuron activities. This paper provides evidence that the…
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two…
The entropy of network ensembles characterizes the amount of information encoded in the network structure, and can be used to quantify network complexity, and the relevance of given structural properties observed in real network datasets…
This paper describes a new entropy-style of equation that may be useful in a general sense, but can be applied to a cognitive model with related processes. The model is based on the human brain, with automatic and distributed pattern…
In stochastic modeling, the excess entropy -- the mutual information shared between a process's past and future -- represents the fundamental lower bound of the memory needed to simulate its dynamics. However, this bound cannot be saturated…
Maximization of the entropy rate is an important issue to design diffusion processes aiming at a well-mixed state. We demonstrate that it is possible to construct maximal-entropy random walks with only local information on the graph…
A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on…
We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…
Depending on context, the term entropy is used for a thermodynamic quantity, a~measure of available choice, a quantity to measure information, or, in the context of statistical inference, a maximum configuration predictor. For systems in…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Entropy maximization procedure has been a general practice in many diverse fields of science to obtain the concomitant probability distributions. The consistent use of the maximization procedure on the other hand requires the probability…
The question of what can be computed, and how efficiently, are at the core of computer science. Not surprisingly, in distributed systems and networking research, an equally fundamental question is what can be computed in a…
We propose a method for inferring entropy production (EP) in high-dimensional stochastic systems, including many-body systems and non-Markovian systems with long memory. Standard techniques for estimating EP become intractable in such…
Optimisation problems in science and engineering typically involve finding the ground state (i.e. the minimum energy configuration) of a cost function with respect to many variables. If the variables are corrupted by noise then this…
Computing the partition function and the marginals of a global probability distribution are two important issues in any probabilistic inference problem. In a previous work, we presented sub-tree based upper and lower bounds on the partition…
Exponential random graph models have attracted significant research attention over the past decades. These models are maximum-entropy ensembles under the constraints that the expected values of a set of graph observables are equal to given…