Related papers: Entropic Inference: some pitfalls and paradoxes we…
Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…
Unlike the Probability Theory based on additivity, Statistical Inference seems to hesitate between "Additivity" and a so-called "Maxitivity" approach. After a brief overview of three types of principles for any (parametric) statistical…
When constructing models of the world, we aim for optimal compressions: models that include as few details as possible while remaining as accurate as possible. But which details -- or features measured in data -- should we choose to include…
The present work shows that the maximum-entropy method can be applied to a sample of neuronal recordings along two different routes: (1) apply to the sample; or (2) apply to a larger, unsampled neuronal population from which the sample is…
Entropy concept was introduced by Clausius 160 years ago, and has been continually enriched, developed and interpreted by the researchers in many different scientific disciplines ever since. Thermodynamics and other scientific disciplines…
The concept of Entropy plays a key role in Information Theory, Statistics, and Machine Learning.This paper introduces a new entropy measure, called the t-entropy, which exploits the concavity of the inverse-tan function. We analytically…
The existing upper and lower bounds between entropy and error are mostly derived through an inequality means without linking to joint distributions. In fact, from either theoretical or application viewpoint, there exists a need to achieve a…
To quantify the complexity of a system, entropy-based methods have received considerable critical attentions in real-world data analysis. Among numerous entropy algorithms, amplitude-based formulas, represented by Sample Entropy, suffer…
Recent years have seen the rise of convolutional neural network techniques in exemplar-based image synthesis. These methods often rely on the minimization of some variational formulation on the image space for which the minimizers are…
The threshold behaviour of the K-Satisfiability problem is studied in the framework of the statistical mechanics of random diluted systems. We find that at the transition the entropy is finite and hence that the transition itself is due to…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
A central question for causal inference is to decide whether a set of correlations fit a given causal structure. In general, this decision problem is computationally infeasible and hence several approaches have emerged that look for…
Bayes' theorem incorporates distinct types of information through the likelihood and prior. Direct observations of state variables enter the likelihood and modify posterior probabilities through consistent updating. Information in terms of…
Entropic uncertainty relations are powerful tools, especially in quantum cryptography. They typically bound the amount of uncertainty a third-party adversary may hold on a measurement outcome as a result of the measurement overlap. However,…
There is a vast body of recent literature on the reliability of communication through noisy channels, the recovery of community structures in the stochastic block model, the limiting behavior of the free entropy in spin glasses and the…
We investigate the entanglement entropy and the information flow of two-dimensional moving mirrors. Here we point out that various mirror trajectories can help to mimic different candidate resolutions to the information loss paradox…
Consider the set of all sequences of $n$ outcomes, each taking one of $m$ values, that satisfy a number of linear constraints. If $m$ is fixed while $n$ increases, most sequences that satisfy the constraints result in frequency vectors…
We take another look at the general problem of selecting a preferred probability measure among those that comply with some given constraints. The dominant role that entropy maximization has obtained in this context is questioned by arguing…
Foundation models excel in stable environments, yet often fail where reliability matters most: medicine, finance, and policy. This Fidelity Paradox is not just a data problem; it is structural. In domains where rules change over time, extra…
This study introduces a novel model that effectively captures asymmetric structures in multivariate contingency tables with ordinal categories. Leveraging the principle of maximum entropy, our approach employs f-divergence to provide a…