Related papers: Shannon and R\'enyi entropy rates of stationary ve…
A method of estimating the joint probability mass function of a pair of discrete random variables is described. This estimator is used to construct the conditional Shannon-R\'eyni-Tsallis entropies estimates. From there almost sure rates of…
A plug-in estimator of entropy is the entropy of the distribution where probabilities of symbols or blocks have been replaced with their relative frequencies in the sample. Consistency and asymptotic unbiasedness of the plug-in estimator…
R\'enyi entropy is a one-parameter generalization of Shannon entropy, which has been used in various fields of physics. Despite its wide applicability, the physical interpretations of the R\'enyi entropy are not widely known. In this paper,…
In this paper, we estimate the Shannon entropy $S(f) = -\E[ \log (f(x))]$ of a one-sided linear process with probability density function $f(x)$. We employ the integral estimator $S_n(f)$, which utilizes the standard kernel density…
Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…
We consider the semi-parametric estimation of a scale parameter of a one-dimensional Gaussian process with known smoothness. We suggest an estimator based on quadratic variations and on the moment method. We provide asymptotic…
We study a quantum process that can be considered as a quantum analogue for the classical Markov process. We specifically construct a version of these processes for free Fermions. For such free Fermionic processes we calculate the entropy…
We introduce a method for quantifying the inherent unpredictability of a continuous-valued time series via an extension of the differential Shannon entropy rate. Our extension, the specific entropy rate, quantifies the amount of predictive…
The idea of a parsing of a stationary process according to a collection of words is introduced, and the basic framework required for the asymptotic analysis of these parsings is presented. We demonstrate how the pointwise ergodic theorem…
Unitary ensembles of large N x N random matrices with a non-Gaussian probability distribution P[H] ~ exp{-TrV[H]} are studied using a theory of polynomials orthogonal with respect to exponential weights. Asymptotically exact expressions for…
The quantum Renyi relative entropies play a prominent role in quantum information theory, finding applications in characterizing error exponents and strong converse exponents for quantum hypothesis testing and quantum communication theory.…
Observables in random tensor theory are polynomials in the entries of a tensor of rank $d$ which are invariant under $U(N)^d$. It is notoriously difficult to evaluate the expectations of such polynomials, even in the Gaussian distribution.…
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the…
Stationary points embedded in the derivatives are often critical for a model to be interpretable and may be considered as key features of interest in many applications. We propose a semiparametric Bayesian model to efficiently infer the…
R\'enyi entropy is an important measure in the context of information theory as a generalization of Shannon entropy. This information measure was often used for uncertainty quantification of dynamical behaviour of stochastic processes. In…
We investigate the small deviation probabilities of a class of very smooth stationary Gaussian processes playing an important role in Bayesian statistical inference. Our calculations are based on the appropriate modification of the entropy…
The R\'{e}nyi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we…
Expressions for (EPI Shannon type) Divergence-Power Inequalities (DPI) in two cases (time-discrete and band-limited time-continuous) of stationary random processes are given. The new expressions connect the divergence rate of the sum of…
We consider the special case of Random Tensor Networks (RTN) endowed with gauge symmetry constraints on each tensor. We compute the R\`enyi entropy for such states and recover the Ryu-Takayanagi (RT) formula in the large bond regime. The…
Methods are presented to evaluate the entropy production rate in stochastic reactive systems. These methods are shown to be consistent with known results from nonequilibrium chemical thermodynamics. Moreover, it is proved that the time…