Related papers: Entropy Rate Estimation for Markov Chains with Lar…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
We propose a novel, tractable latent state inference scheme for Markov jump processes, for which exact inference is often intractable. Our approach is based on an entropic matching framework that can be embedded into the well-known…
Entropy is a measure of the randomness of a system. Estimating the entropy of a quantum state is a basic problem in quantum information. In this paper, we introduce a time-efficient quantum approach to estimating the von Neumann entropy…
Shannon and Renyi entropies are quantitative measures of uncertainty in a data set. They are developed by Renyi in the context of entropy theory. These measures have been studied in the case of the multivariate t-distributions. We extend…
In this paper, we estimate the Shannon entropy $S(f) = -\E[ \log (f(x))]$ of a one-sided linear process with probability density function $f(x)$. We employ the integral estimator $S_n(f)$, which utilizes the standard kernel density…
Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system…
The ensemble of random Markov matrices is introduced as a set of Markov or stochastic matrices with the maximal Shannon entropy. The statistical properties of the stationary distribution pi, the average entropy growth rate $h$ and the…
We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. We review large deviations techniques useful…
Estimating entropy production from real observation data can be difficult due to finite resolution in both space and time and finite measurement statistics. We characterize the statistical error introduced by finite sample size and compare…
The mixing time of a Markov chain determines how fast the iterates of the Markov chain converge to the stationary distribution; however, it does not control the dependencies between samples along the Markov chain. In this paper, we study…
We define the projected entropy S(T) at a given temperature T in the context of an Ising model transition matrix calculation as the entropy associated with the distribution of Markov chain realizations in energy-magnetization, E-H, space.…
We consider spin systems with nearest-neighbor interactions on an $n$-vertex $d$-dimensional cube of the integer lattice graph $\mathbb{Z}^d$. We study the effects that exponential decay with distance of spin correlations, specifically the…
In this paper we analyze the probability of consistency of sensor data distribution systems (SDDS), and determine suitable evaluation models. This problem is typically difficult, since a reliable model taking into account all parameters and…
In this paper, we consider the information content of maximum ranked set sampling procedure with unequal samples (MRSSU) in terms of Tsallis entropy which is a nonadditive generalization of Shannon entropy. We obtain several results of…
We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax…
The exact treatment of Markovian models of complex systems requires knowledge of probability distributions exponentially large in the number of components $n$. Mean-field approximations provide an effective reduction in complexity of the…
Entropy has been a common index to quantify the complexity of time series in a variety of fields. Here, we introduce increment entropy to measure the complexity of time series in which each increment is mapped into a word of two letters,…
We explore a supervised machine learning approach to estimate the entanglement entropy of multi-qubit systems from few experimental samples. We put a particular focus on estimating both aleatoric and epistemic uncertainty of the network's…
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…