相关论文: Word-Valued Sources: an Ergodic Theorem, an AEP an…

We show that the typical coordinate-wise encoding of multivariate ergodic source into prescribed alphabets has the entropy profile close to the convolution of the entropy profile of the source and the modular polymatroid that is determined…

Suppose a string $X_1^n=(X_1,X_2,...,X_n)$ generated by a memoryless source $(X_n)_{n\geq 1}$ with distribution $P$ is to be compressed with distortion no greater than $D\geq 0$, using a memoryless random codebook with distribution $Q$. The…

It is demonstrated how to represent asymptotically mean stationary (AMS) random sources with values in standard spaces as mixtures of ergodic AMS sources. This an extension of the well known decomposition of stationary sources which has…

In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth…

An antidictionary code is a lossless compression algorithm using an antidictionary which is a set of minimal words that do not occur as substrings in an input string. The code was proposed by Crochemore et al. in 2000, and its asymptotic…

Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…

Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…

Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…

The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…

We study ergodic-theoretic properties of coded shift spaces. A coded shift space is defined as a closure of all bi-infinite concatenations of words from a fixed countable generating set. We derive sufficient conditions for the uniqueness of…

We extend the data compression theorem to the case of ergodic quantum information sources. Moreover, we provide an asymptotically optimal compression scheme which is based on the concept of high probability subspaces. The rate of this…

The idea of a parsing of a stationary process according to a collection of words is introduced, and the basic framework required for the asymptotic analysis of these parsings is presented. We demonstrate how the pointwise ergodic theorem…

A power-free language is characterized by the number of symbols used and a limit on how many times a block of symbols can repeat consecutively. For certain values of these parameters, it is known that the number of legal words grows…

We show that a certain type of quasi finite, conservative, ergodic, measure preserving transformation always has a maximal zero entropy factor, generated by predictable sets. We also construct a conservative, ergodic, measure preserving…

We prove existence of asymptotic entropy of random walks on regular languages over a finite alphabet and we give formulas for it. Furthermore, we show that the entropy varies real-analytically in terms of probability measures of constant…

A probabilistic source is defined as the set of infinite words (over a given denumerable alphabet) endowed with a probability $\mu$. The paper deals with general binary sources where the distribution of any symbol (0 or 1) may depend on an…

We construct multiperiodic processes -- a simple example of stationary ergodic (but not mixing) processes over natural numbers that enjoy the vanishing entropy rate under a mild condition. Multiperiodic processes are supported on randomly…

Entropy production (EP) is known as a fundamental quantity for measuring the irreversibility of processes in thermal equilibrium and states far from equilibrium. In stochastic thermodynamics, the EP becomes more visible in terms of the…

As we discuss, a stationary stochastic process is nonergodic when a random persistent topic can be detected in the infinite random text sampled from the process, whereas we call the process strongly nonergodic when an infinite sequence of…

State-of-the-art language generation models can degenerate when applied to open-ended generation problems such as text completion, story generation, or dialog modeling. This degeneration usually shows up in the form of incoherence, lack of…