Related papers: Maximum-entropy from the probability calculus: exc…
Bayesian hierarchical models are frequently used in practical data analysis contexts. One interpretation of these models is that they provide an indirect way of assigning a prior for unknown parameters, through the introduction of…
Moment-closure methods are popular tools to simplify the mathematical analysis of stochastic models defined on networks, in which high dimensional joint distributions are approximated (often by some heuristic argument) as functions of lower…
Entropy serves as a central observable which indicates uncertainty in many chemical, thermodynamical, biological and ecological systems, and the principle of the maximum entropy (MaxEnt) is widely supported in natural science. Recently,…
For the purpose of causal inference we employ a stochastic model of the data generating process, utilizing individual propensity probabilities for the treatment, and also individual and counterfactual prognosis probabilities for the…
Exponential families form the backbone of modern statistics and machine learning, but textbooks seldom derive them from first principles in an accessible way. Although minimal sufficiency and the principle of maximum entropy, originating in…
The maximum-entropy sampling problem is a fundamental and challenging combinatorial-optimization problem, with application in spatial statistics. It asks to find a maximum-determinant order-$s$ principal submatrix of an order-$n$ covariance…
We demonstrate and characterize a first-principles approach to modeling the mass action dynamics of metabolism. Starting from a basic definition of entropy expressed as a multinomial probability density using Boltzmann probabilities with…
This note aims to demonstrate that performing maximum-likelihood estimation for a mixture model is equivalent to minimizing over the parameters an optimal transport problem with entropic regularization. The objective is pedagogical: we seek…
We derive an efficient method to calculate exceedance probabilities (EP) for the Dirichlet distribution when the number of event types is larger than two. Also, we present an intuitive application of Dirichlet EPs and compare our method to…
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example.…
We extend de Finetti's [Ann. Inst. H. Poincar\'{e} 7 (1937) 1--68] notion of exchangeability to finite and countable sequences of variables, when a subject's beliefs about them are modelled using coherent lower previsions rather than…
In this tutorial we review the essential arguments behing entropic inference. We focus on the epistemological notion of information and its relation to the Bayesian beliefs of rational agents. The problem of updating from a prior to a…
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
In this work we develop on the recently suggested concept of superstatistics [C. Beck and E.G.D. Cohen, Physica A {\bf 322}, 267 (2003)], face the problem of devising a viable way for estimating the correct statistics for a system in…
We characterize the existence of the maximum likelihood estimator for discrete exponential families. Our criterion is simple to apply as we show in various settings, most notably for exponential models of random graphs. As an application,…
To efficiently evaluate system reliability based on Monte Carlo simulation, importance sampling is used widely. The optimal importance sampling density was derived in 1950s for the deterministic simulation model, which maps an input to an…
The Principle of Maximum Entropy, a powerful and general method for inferring the distribution function given a set of constraints, is applied to deduce the overall distribution of 3D plasmoids (flux ropes/tubes) for systems where resistive…
Maximum entropy models provide the least constrained probability distributions that reproduce statistical properties of experimental datasets. In this work we characterize the learning dynamics that maximizes the log-likelihood in the case…
Recommendations based on behavioral data may be faced with ambiguous statistical evidence. We consider the case of association rules, relevant e.g.~for query and product recommendations. For example: Suppose that a customer belongs to…
Probability distributions of money, income, and energy consumption per capita are studied for ensembles of economic agents. The principle of entropy maximization for partitioning of a limited resource gives exponential distributions for the…