相关论文: Large Alphabets and Incompressibility

In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors introduced the duality between values (words) and orderings (permutations) as a basis to discuss the relationship between information theoretic measures for…

In this paper we present results on asymptotic characteristics of multivariate function classes in the uniform norm. Our main interest is the approximation of functions with mixed smoothness parameter not larger than $1/2$. Our focus will…

We prove lower bounds for the randomized approximation of the embedding $\ell_1^m \rightarrow \ell_\infty^m$ based on algorithms that use arbitrary linear (hence non-adaptive) information provided by a (randomized) measurement matrix $N \in…

In 2002, Kamae and Zamboni introduced maximal pattern complexity and determined that any aperiodic sequence must have maximal pattern complexity at least $2k$. In 2006, Kamae and Rao examined the maximal pattern complexity of sequences over…

In this work, we study the problem of finding the asymptotic growth rate of the number of of $d$-dimensional arrays with side length $n$ over a given alphabet which avoid a list of one-dimensional "forbidden" words along all cardinal…

Direct estimates between linear or nonlinear Kolmogorov widths and entropy numbers are presented. These estimates are derived using the recently introduced Lipschitz widths. Applications for m-term approximation are obtained.

Breaking of ensemble equivalence between the microcanonical ensemble and the canonical ensemble may occur for random graphs whose size tends to infinity, and is signaled by a non-zero specific relative entropy of the two ensembles. In [3]…

We study the asymptotics and fine-scale behavior of quantitative combinatorial measures of infinite words and related dynamical and algebraic structures. We construct infinite recurrent words $w$ whose complexity functions $p_w(n)$ are…

Large-alphabet strings are common in scenarios such as information retrieval and natural-language processing. The efficient storage and processing of such strings usually introduces several challenges that are not witnessed in…

This paper investigates achievable information rates and error exponents of mismatched decoding when the channel belongs to the class of channels that are close to the decoding metric in terms of relative entropy. For both discrete- and…

We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…

We respond to the comment by Crutchfield, Feldman and Shalizi and that by Binder and Perry, pointing out that there may be many maximum entropies, and therefore ``disorders'' and ``simple complexities''. Which ones are appropriate depend on…

We study in which way Kolmogorov complexity and instance complexity affect properties of r.e. sets. We show that the well-known 2log n upper bound on the Kolmogorov complexity of initial segments of r.e.\ sets is optimal and characterize…

We relate the computational complexity of finite strings to universal representations of their underlying symmetries. First, Boolean functions are classified using the universal covering topologies of the circuits which enumerate them. A…

Recent work of Acharya et al. (NeurIPS 2019) showed how to estimate the entropy of a distribution $\mathcal D$ over an alphabet of size $k$ up to $\pm\epsilon$ additive error by streaming over $(k/\epsilon^3) \cdot…

The problem addressed concerns the determination of the average number of successive attempts of guessing a word of a certain length consisting of letters with given probabilities of occurrence. Both first- and second-order approximations…

Consider a binary string $x$ of length $n$ whose Kolmogorov complexity is $\alpha n$ for some $\alpha<1$. We want to increase the complexity of $x$ by changing a small fraction of bits in $x$. This is always possible: Buhrman, Fortnow,…

A practical measure for the complexity of sequences of symbols (``strings'') is introduced that is rooted in automata theory but avoids the problems of Kolmogorov-Chaitin complexity. This physical complexity can be estimated for ensembles…

This paper investigates model reduction methods for efficiently approximating the solution of parameter-dependent PDEs with a multi-parameter vector $\vec{\mu} \in \mathbb{R}^p$. In cases where the Kolmogorov $N$-width decays fast enough,…

We consider the problem of reducing a first-order Markov chain on a large alphabet to a higher-order Markov chain on a small alphabet. We present information-theoretic cost functions that are related to predictability and lumpability, show…