Related papers: Entropy Rate Bounds via Second-Order Statistics
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…
The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter epsilon. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
We consider the stochastic ranking process with the jump times of the particles determined by Poisson random measures. We prove that the joint empirical distribution of scaled position and intensity measure converges almost surely in the…
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
We provide an upper bound on the maximal entropy rate at which the entropy of the expected density operator of a given ensemble of two states changes under nonlocal unitary evolution. A large class of entropy measures in considered, which…
We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…
Hybrid inverse problems are based on the interplay of two types of waves, in order to allow for imaging with both high resolution and high contrast. The inversion procedure often consists of two steps: first, internal measurements involving…
Quantum many-body states that frequently appear in physics often obey an entropy scaling law, meaning that an entanglement entropy of a subsystem can be expressed as a sum of terms that scale linearly with its volume and area, plus a…
Rooted trees with probabilities are used to analyze properties of a variable length code. A bound is derived on the difference between the entropy rates of the code and a memoryless source. The bound is in terms of normalized informational…
We consider a generic class of stochastic particle-based models whose state at an instant in time is described by a set of continuous degrees of freedom (e.g. positions), and the length of this set changes stochastically in time due to…
We propose an algorithm to numerically determined whether a second-order linear PDE problem satisfying a Garding inequality is well-posed. This algorithm further provides a lower bound to the inf-sup constant of the weak formulation, which…
We present a novel scheme for the appearance of Stochastic Resonance when the dynamics of a Brownian particle takes place in a confined medium. The presence of uneven boundaries, giving rise to an entropic contribution to the potential, may…
Consider a first-come, first-served single server queue with an initial workload $x>0$ and customers who arrive according to an inhomogeneous Poisson process with rate function $\lambda:[0,\infty)\rightarrow[0,\lambda_h ]$ for some…
We discuss the limiting spectral density of real symmetric random matrices. Other than in standard random matrix theory the upper diagonal entries are not assumed to be independent, but we will fill them with the entries of a stochastic…
In the context of control and estimation under information constraints, restoration entropy measures the minimal required data rate above which the state of a system can be estimated so that the estimation quality does not degrade over time…
A statistical model of discrete finite length random processes with negative power law spectral densities is presented. The definition of terms is followed by a description of the spectral density trend. An algorithmic construction of…