Related papers: On Maximum Entropy and Inference
Advanced inference techniques allow one to reconstruct the pattern of interaction from high dimensional data sets. We focus here on the statistical properties of inferred models and argue that inference procedures are likely to yield models…
Maximal inequalities refer to bounds on expected values of the supremum of averages of random variables over a collection. They play a crucial role in the study of non-parametric and high-dimensional estimators, and especially in the study…
The science of cities is a relatively new and interdisciplinary topic. It borrows techniques from agent-based modeling, stochastic processes, and partial differential equations. However, how the cities rise and fall, how they evolve, and…
In the analysis of any type of system, granting maximum information extraction from its data is non-trivial. Confidence in successful information extraction typically builds on prior knowledge of the studied system or on the user's…
How does the information flow between different brain regions during various stimuli? This is the question we aim to address by studying complex cognitive paradigms in terms of Information Theory. To assess creativity and the emergence of…
The kinematics and dynamics of deterministic physical systems have been a foundation of our understanding of the world since Galileo and Newton. For real systems, however, uncertainty is largely present via external forces such as friction…
We give an interpretation of the Maximum Entropy (MaxEnt) Principle in game-theoretic terms. Based on this interpretation, we make a formal distinction between different ways of {em applying/} Maximum Entropy distributions. MaxEnt has…
Transfer entropy is used to establish a measure of causal relationships between two variables. Symbolic transfer entropy, as an estimation method for transfer entropy, is widely applied due to its robustness against non-stationarity. This…
Artificial intelligence models and methods commonly lack causal interpretability. Despite the advancements in interpretable machine learning (IML) methods, they frequently assign importance to features which lack causal influence on the…
We study the problem of identifying the causal relationship between two discrete random variables from observational data. We recently proposed a novel framework called entropic causality that works in a very general functional model but…
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
The concept of entropy connects the number of possible configurations with the number of variables in large stochastic systems. Independent or weakly interacting variables render the number of configurations scale exponentially with the…
Although instantaneous interactions are unphysical, a large variety of maximum entropy statistical inference methods match the model-inferred and the empirically-measured equal-time correlation functions. Focusing on collective motion of…
Configurational entropy is an important factor in the free energy change of many macromolecular recognition and binding processes, and has been intensively studied. Despite great progresses that have been made, the global sampling remains…
In this paper we examine the problem of inference in Bayesian Networks with discrete random variables that have very large or even unbounded domains. For example, in a domain where we are trying to identify a person, we may have variables…
We investigate activities that have different periods of duration. We define the profit intensity as a measure of this economic category. The profit intensity in a repeated trading has a unique property of attaining its maximum at a fixed…
The method of maximum entropy has proven to be a rather powerful way to solve the inverse problem consisting of determining a probability density $f_S(s)$ on $[0,\infty)$ from the knowledge of the expected value of a few generalized…
We give some results relating asymptotic characterisations of maximum entropy probability measures to characterisations of Bayes optimal classifiers. Our main theorems show that maximum entropy is a universally Bayes optimal decision rule…
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's…
Econophysics, is based on the premise that some ideas and methods from physics can be applied to economic situations. We intend to show in this paper how a physics concept such as entropy can be applied to an economic problem. In so doing,…