Related papers: Patterns of i.i.d. Sequences and Their Entropy - P…

Labeled infinite trees provide combinatorial interpretations for many integer sequences generated by nested recurrence relations. Typically, such sequences are monotone increasing. Several of these sequences also have straightforward…

Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…

In this paper we consider the class of anti-uniform Huffman codes and derive tight lower and upper bounds on the average length, entropy, and redundancy of such codes in terms of the alphabet size of the source. The Fibonacci distributions…

The problem of reconstructing a sequence of independent and identically distributed symbols from a set of equal size, consecutive, fragments, as well as a dependent reference sequence, is considered. First, in the regime in which the…

In index coding, a server broadcasts multiple messages to their respective receivers, each with some side information that can be utilized to reduce the amount of communication from the server. Distributed index coding is an extension of…

Graphs are widely used for describing systems made up of many interacting components and for understanding the structure of their interactions. Various statistical models exist, which describe this structure as the result of a combination…

Abstract: In this article, we will analyze in detail the coding limit of an individual sequence by introducing the latest developments brought by the Set Shaping Theory. This new theory made us realize that there is a huge difference…

We prove large (and moderate) deviations for a class of linear combinations of spacings generated by i.i.d. exponentially distributed random variables. We allow a wide class of coefficients which can be expressed in terms of continuous…

Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…

This paper describes a design that can be used for Explainable AI. The lower level is a nested ensemble of patterns created by self-organisation. The upper level is a hierarchical tree, where nodes are linked through individual concepts, so…

In this work we consider the task of relaxing the i.i.d assumption in pattern recognition (or classification), aiming to make existing learning algorithms applicable to a wider range of tasks. Pattern recognition is guessing a discrete…

Batch codes are a useful notion of locality for error correcting codes, originally introduced in the context of distributed storage and cryptography. Many constructions of batch codes have been given, but few lower bound (limitation)…

In this paper we show the distributions of sliding block patterns for Bernoulli processes with finite alphabet, which is not based on the induction on sample size. We show a new inclusion-exclusion formula in multivariate generating…

A binary string transmitted via a memoryless i.i.d. deletion channel is received as a subsequence of the original input. From this, one obtains a posterior distribution on the channel input, corresponding to a set of candidate…

We consider a wide class of linear stochastic problems driven off the equilibrium by a multiplicative asymmetric force. The force brakes detailed balance, maintained otherwise, thus producing entropy. The large deviation function of the…

The classical setting of community detection consists of networks exhibiting a clustered structure. To more accurately model real systems we consider a class of networks (i) whose edges may carry labels and (ii) which may lack a clustered…

The traditional approach to the quantitative study of segregation is to employ indices that are selected by ``desirable properties''. Here, we detail how information theory underpins entropy-based indices and demonstrate how desirable…

Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…

Entropy production is often interpreted as a proxy for microscopic disorder or environmental roughness in stochastic systems. We test this interpretation using controlled simulations of overdamped stochastic dynamics on curved surfaces in…

Recurrence plots provide a graphical representation of the recurrent patterns in a timeseries, the quantification of which is a relatively new field. Here we derive analytical expressions which relate the values of key statistics, notably…