Related papers: Shannon Weights for binary dynamical recurrent sou…
We present a study of the equilibration process of nonequilibrium systems by means of molecular dynamics simulation technique. The nonequilibrium conditions are achieved in systems by defining velocity components of the constituent atoms…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
Entropy has been a common index to quantify the complexity of time series in a variety of fields. Here, we introduce increment entropy to measure the complexity of time series in which each increment is mapped into a word of two letters,…
In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k…
We suggest to construct infinite stochastic binary sequences by associating one of the two symbols of the sequence with the renewal times of an underlying renewal process. Focusing on stationary binary sequences corresponding to delayed…
The fractional order generalization of Shannon entropy proposed by Ubriaco has been studied for discrete distributions. In the current paper, we conduct a detailed study of the continuous analogue of this entropy termed as fractional…
With the advent of massive data outputs at a regular rate, admittedly, signal processing technology plays an increasingly key role. Nowadays, signals are not merely restricted to physical sources, they have been extended to digital sources…
A transmitter observing a sequence of independent and identically distributed random variables seeks to keep a receiver updated about its latest observations. The receiver need not be apprised about each symbol seen by the transmitter, but…
Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system…
We consider the problem of when a symbolic dynamical system supports a Borel probability measure that is invariant under every element of its automorphism group. It follows readily from a classical result of Parry that the full shift on…
Any unconstrained information inequality in three or fewer random variables can be written as a linear combination of instances of Shannon's inequality I(A;B|C) >= 0 . Such inequalities are sometimes referred to as "Shannon" inequalities.…
Consider a sequence $X^n$ of length $n$ emitted by a Discrete Memoryless Source (DMS) with unknown distribution $p_X$. The objective is to construct a lossless source code that maps $X^n$ to a sequence $\widehat{Y}^m$ of length $m$ that is…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
Shannon entropy is often a quantity of interest to linguists studying the communicative capacity of human language. However, entropy must typically be estimated from observed data because researchers do not have access to the underlying…
In this paper we derive explicit formulas of the R\'enyi information, Shannon entropy and Song measure for the invariant density of one dimensional ergodic diffusion processes. In particular, the diffusion models considered include the…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
The extraction of spatio-temporal coherence in high-dimensional, chaotic, non-linear dynamical systems, such as turbulent flows, remains a fundamental challenge in physics, mathematics and engineering. In this work, we employ Shannon…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
We study the behavior of a nonlinear semiclassical system using Shannon entropy and two approaches to statistical complexity. These systems involve the interaction between classical variables (representing the environment) and quantum ones.…