Related papers: On analytic properties of entropy rate

Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…

Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…

The entropy of a hierarchical network topology in an ensemble of sparse random networks with "hidden variables" associated to its nodes, is the log-likelihood that a given network topology is present in the chosen ensemble.We obtain a…

We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of…

Topological entropy is a widely studied indicator of chaos in topological dynamics. Here we give a generalized definition of topological entropy which may be applied to set-valued functions. We demonstrate that some of the well-known…

Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…

Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…

The interrelationships of the fundamental biological processes natural selection, mutation, and stochastic drift are quantified by the entropy rate of Moran processes with mutation, measuring the long-run variation of a Markov process. The…

The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates…

We consider random walks on a non-elementary hyperbolic group endowed with a word distance. To a probability measure on the group are associated two numerical quantities, the rate of escape and the entropy. On the set of admissible…

We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…

The amount of information generated by a discrete time stochastic processes in a single step can be quantified by the entropy rate. We investigate the differences between two discrete time walk models, the discrete time quantum walk and the…

We prove existence of asymptotic entropy of random walks on regular languages over a finite alphabet and we give formulas for it. Furthermore, we show that the entropy varies real-analytically in terms of probability measures of constant…

Suppose we are given the free product V of a finite family of finite or countable sets. We consider a transient random walk on the free product arising naturally from a convex combination of random walks on the free factors. We prove the…

Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…

Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…

We study properties of ridge functions $f(x)=g(a\cdot x)$ in high dimensions $d$ from the viewpoint of approximation theory. The considered function classes consist of ridge functions such that the profile $g$ is a member of a univariate…

The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically,…

Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…

Extropy, a complementary dual of entropy, (proposed by Lad et al. \cite{lad2015extropy} in 2015) has attracted considerable interest from the research community. In this study, we focus on discrete random variables and define conditional…