Related papers: MEP-Net: Generating Solutions to Scientific Proble…
In this work we develop on the recently suggested concept of superstatistics [C. Beck and E.G.D. Cohen, Physica A {\bf 322}, 267 (2003)], face the problem of devising a viable way for estimating the correct statistics for a system in…
Maximum entropy models are the least structured probability distributions that exactly reproduce a chosen set of statistics measured in an interacting network. Here we use this principle to construct probabilistic models which describe the…
We describe and develop a close relationship between two problems that have customarily been regarded as distinct: that of maximizing entropy, and that of minimizing worst-case expected loss. Using a formulation grounded in the equilibrium…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
Generating samples from limited information is a fundamental problem across scientific domains. Classical maximum entropy methods provide principled uncertainty quantification from moment constraints but require sampling via MCMC or…
Understanding the structural complexity and predictability of complex networks is a central challenge in network science. Although recent studies have revealed a relationship between compression-based entropy and link prediction…
With the widespread success of deep neural networks in science and technology, it is becoming increasingly important to quantify the uncertainty of the predictions produced by deep learning. In this paper, we introduce a new method that…
Evidence theory is that the extension of probability can better deal with unknowns and inaccurate information. Uncertainty measurement plays a vital role in both evidence theory and probability theory. Approximate Entropy (ApEn) is proposed…
One of the central challenges in modern machine learning is understanding how neural networks generalize knowledge learned from training data to unseen test data. While numerous empirical techniques have been proposed to improve…
We explore the use of the method of Maximum Entropy (ME) as a technique to generate approximations. In a first use of the ME method the "exact" canonical probability distribution of a fluid is approximated by that of a fluid of hard…
Nature is full of random networks of complex topology describing such apparently disparate systems as biological, economical or informatical ones. Their most characteristic feature is the apparent scale-free character of interconnections…
We propose a novel molecular computing scheme for statistical inference. We focus on the much-studied statistical inference problem of computing maximum likelihood estimators for log-linear models. Our scheme takes log-linear models to…
This study examines a new formulation of non-equilibrium thermodynamics, which gives a conditional derivation of the ``maximum entropy production'' (MEP) principle for flow and/or chemical reaction systems at steady state. The analysis uses…
Modelling a complex system is almost invariably a challenging task. The incorporation of experimental observations can be used to improve the quality of a model, and thus to obtain better predictions about the behavior of the corresponding…
Obtaining high certainty in predictive models is crucial for making informed and trustworthy decisions in many scientific and engineering domains. However, extensive experimentation required for model accuracy can be both costly and…
Deep neural networks have amply demonstrated their prowess but estimating the reliability of their predictions remains challenging. Deep Ensembles are widely considered as being one of the best methods for generating uncertainty estimates…
Simplified mechanistic models in ecology have been criticized for the fact that a good fit to data does not imply the mechanism is true: pattern does not equal process. In parallel, the maximum entropy principle (MaxEnt) has been applied in…
We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural…
This paper establishes the minimum entropy principle (MEP) for the relativistic Euler equations with a broad class of equations of state (EOSs) and addresses the challenge of preserving the local version of the discovered MEP in high-order…
Jaynes' maximum entropy (MaxEnt) principle was recently used to give a conditional, local derivation of the ``maximum entropy production'' (MEP) principle, which states that a flow system with fixed flow(s) or gradient(s) will converge to a…