Related papers: Entropy Rate Bounds via Second-Order Statistics
According to the universal entropy bound, the entropy (and hence information capacity) of a complete weakly self-gravitating physical system can be bounded exclusively in terms of its circumscribing radius and total gravitating energy. The…
The amount of information generated by a discrete time stochastic processes in a single step can be quantified by the entropy rate. We investigate the differences between two discrete time walk models, the discrete time quantum walk and the…
In this paper, we derive non-asymptotic achievability and converse bounds on the random number generation with/without side-information. Our bounds are efficiently computable in the sense that the computational complexity does not depend on…
A device-independent randomness expansion protocol aims to take an initial random string and generate a longer one, where the security of the protocol does not rely on knowing the inner workings of the devices used to run it. In order to do…
Statistics of stochastic processes are crucially influenced by the boundary conditions. In one spatial dimension, for example, the first passage time distribution in semi-infinite space (one absorbing boundary) is markedly different from…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
We provide a thorough study of stability of the 1-D continuity equation, which models many physical conservation laws. In our system-theoretic perspective, the velocity is considered to be an input. An additional input appears in the…
The Steepest Entropy Ascent approach is considered and applied to few-state systems. When the Hamiltonian of the system is time dependent, the principle of maximum entropy production can still be exploited; arguments to support this fact…
Grammar compression represents a string as a context free grammar. Achieving compression requires encoding such grammar as a binary string; there are a few commonly used encodings. We bound the size of practically used encodings for several…
In this second part, we analyze the dissipation properties of Generalized Poisson-Kac (GPK) processes, considering the decay of suitable $L^2$-norms and the definition of entropy functions. In both cases, consistent energy dissipation and…
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
Consider an M/M/$s$ queue with the additional feature that the arrival rate is a random variable of which only the mean, variance, and range are known. Using semi-infinite linear programming and duality theory for moment problems, we…
We consider the use of the well-known dual capacity bounding technique for deriving upper bounds on the capacity of indecomposable finite-state channels (FSCs) with finite input and output alphabets. In this technique, capacity upper bounds…
We consider discrete stochastic processes, modeled by classical master equations, on networks. The temporal growth of the lack of information about the system is captured by its non-equilibrium entropy, defined via the transition…
In this paper, we examine the fundamental performance limitations in the control of stochastic dynamical systems; more specifically, we derive generic $\mathcal{L}_p$ bounds that hold for any causal (stabilizing) controllers and any…
Moving boundary problems allow to model systems with phase transition at an inner boundary. Driven by problems in economics and finance, in particular modeling of limit order books, we consider a stochastic and non-linear extension of the…
Estimating the p-th frequency moment of data stream is a very heavily studied problem. The problem is actually trivial when p = 1, assuming the strict Turnstile model. The sample complexity of our proposed algorithm is essentially O(1) near…
We derive Bose-Einstein statistics and Fermi-Dirac statistics by Principle of Maximum Entropy applied to two families of entropy functions different from the Boltzmann-Gibbs-Shannon entropy. These entropy functions are identified with…
Two-timescale Stochastic Approximation (SA) algorithms are widely used in Reinforcement Learning (RL). Their iterates have two parts that are updated using distinct stepsizes. In this work, we develop a novel recipe for their finite sample…
In this work we introduce a method for estimating entropy rate and entropy production rate from finite symbolic time series. From the point of view of statistics, estimating entropy from a finite series can be interpreted as a problem of…