Related papers: Entropy Rate Bounds via Second-Order Statistics
We define the topological entropy per unit volume in parabolic PDE's such as the complex Ginzburg-Landau equation, and show that it exists, and is bounded by the upper Hausdorff dimension times the maximal expansion rate. We then give a…
Stochastic epidemic models on networks are inherently high-dimensional and the resulting exact models are intractable numerically even for modest network sizes. Mean-field models provide an alternative but can only capture average…
Information theory on a time-discrete setting in the framework of time series analysis is generalized to the time-continuous case. Considerations of the Roessler and Lorenz dynamics as well as the Ornstein-Uhlenbeck process yield for…
The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which detailed balance and time-reversal symmetry are broken. Despite…
We derive the first four terms in a series for the order paramater (the stationary activity density rho) in the supercritical regime of a one-dimensional stochastic sandpile; in the two-dimensional case the first three terms are reported.…
Stochastic dominance (SD) provides a quantile-based partial ordering of random variables and has broad applications. Its extension to multivariate settings, however, is challenging due to the lack of a canonical ordering in $\mathbb{R}^d$…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
In this paper we present a new approach for tightening upper bounds on the partition function. Our upper bounds are based on fractional covering bounds on the entropy function, and result in a concave program to compute these bounds and a…
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…
We develop a well-posedness theory for second order systems in bounded domains where boundary phenomena like glancing and surface waves play an important role. Attempts have previously been made to write a second order system consisting of…
In this paper we determine quantitative stability bounds for the Hessian of entropic potentials, \ie, the dual solution to the entropic optimal transport problem. To the authors' knowledge this is the first work addressing this second-order…
We consider the entanglement entropy in critical one-dimensional quantum systems with open boundary conditions. We show that the second R\'enyi entropy of an interval away from the boundary can be computed exactly, provided the same…
In this work, we study the generalization capability of algorithms from an information-theoretic perspective. It has been shown that the expected generalization error of an algorithm is bounded from above by a function of the relative…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
We design an algorithm which finds an $\epsilon$-approximate stationary point (with $\|\nabla F(x)\|\le \epsilon$) using $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector products, matching guarantees that were previously available…
In the present paper we consider the problem of estimating the multidimensional power spectral density which describes a second-order stationary random field from a finite number of covariance and generalized cepstral coefficients. The…
Stochastic boundary conditions for interactions with a particle reservoir are discussed in many-particle systems. We introduce the boundary conditions with the injection rate and the momentum distribution of particles coming from a particle…
In a pair of recent articles [PRL 105 (2010) 041302 - arXiv:1005.1132; JHEP 1103 (2011) 056 - arXiv:1012.2867] two of the current authors have developed an entropy bound for equilibrium uncollapsed matter using only classical general…
Strict stationarity is a common assumption used in the time series literature in order to derive asymptotic distributional results for second-order statistics, like sample autocovariances and sample autocorrelations. Focusing on weak…