Related papers: Variational Principle underlying Scale Invariant S…
We demonstrate with a thought experiment that fitness-based population dynamical approaches to evolution are not able to make quantitative, falsifiable predictions about the long-term behavior of evolutionary systems. A key characteristic…
Starting from the geometrical interpretation of the R\'enyi entropy, we introduce further extensive generalizations and study their properties. In particular, we found the probability distribution function obtained by the MaxEnt principle…
We present a latent variable model for classification that provides a novel probabilistic interpretation of neural network softmax classifiers. We derive a variational objective to train the model, analogous to the evidence lower bound…
The standard formulation of thermostatistics, being based on the Boltzmann-Gibbs distribution and logarithmic Shannon entropy, describes idealized uncorrelated systems with extensive energies and short-range interactions. In this letter, we…
We describe a particular approach for the construction of a nonequilibrium statistical ensemble formalism for the treatment of dissipative many-body systems. This is the so-called Nonequilibrium Statistical Operator Method, based on the…
Multivariate functional data present theoretical and practical complications which are not found in univariate functional data. One of these is a situation where the component functions of multivariate functional data are positive and are…
The theorem of Shannon-McMillan-Breiman states that for every generating partition on an ergodic system of finite entropy the exponential decay rate of the measure of cylinder sets equals the metric entropy almost everywhere. In addition…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
We introduce a family of scale-invariant entropy statistics derived from logarithmically aggregated distance distributions of point processes, with prime numbers serving as a motivating example. The construction associates to each finite…
We study the Ergodic Properties of Random Walks in stationary ergodic environments without uniform ellipticity under a minimal assumption. There are two main components in our work. The first step is to adopt the arguments of Lawler to…
This work develops problem statements related to encoders and autoencoders with the goal of elucidating variational formulations and establishing clear connections to information-theoretic concepts. Specifically, four problems with varying…
In this study, the cumulative effect of the empirical probability distribution of a random variable is identified as a factor that amplifies the occurrence of extreme events in datasets. To quantify this observation, a corresponding…
In the context of statistical learning, the Information Bottleneck method seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description…
A path information is defined in connection with different possible paths of irregular dynamic systems moving in its phase space between two points. On the basis of the assumption that the paths are physically differentiated by their…
We introduce a new dynamical indicator of stability based on the Extreme Value statistics showing that it provides an insight on the local stability properties of dynamical systems. The indicator perform faster than other based on the…
Complex adaptive systems (CAS) can be described as systems of information flows dynamically interacting across scales in order to adapt and survive. CAS often consist of many components that work towards a shared goal, and interact across…
The concept of Shannon Entropy for probability distributions and associated Maximum Entropy Principle are extended here to the concepts of Relative Divergence of one Grading Function from another and Maximum Relative Divergence Principle…