Related papers: Patterns of i.i.d. Sequences and Their Entropy - P…

In this paper, we propose {\em distributed network compression via memory}. We consider two spatially separated sources with correlated unknown source parameters. We wish to study the universal compression of a sequence of length $n$ from…

PAE cannot be made a basis for either a generalized statistical mechanics or a generalized information theory. Either statistical independence must be waived, or the expression of the averaged conditional probability as the difference…

A formalism is proposed to describe entangled quantum histories, and their entanglement entropy. We define a history vector, living in a tensor space with basis elements corresponding to the allowed histories, i.e. histories with…

We consider discrete stochastic processes, modeled by classical master equations, on networks. The temporal growth of the lack of information about the system is captured by its non-equilibrium entropy, defined via the transition…

We introduce an algorithm for the uniform generation of infinite runs in concurrent systems under a partial order probabilistic semantics. We work with trace monoids as concurrency models. The algorithm outputs on-the-fly approximations of…

Network coding theory studies the transmission of information in networks whose vertices may perform nontrivial encoding and decoding operations on data as it passes through the network. The main approach to deciding the feasibility of…

Explainable artificial intelligence has rapidly emerged since lawmakers have started requiring interpretable models for safety-critical domains. Concept-based neural networks have arisen as explainable-by-design methods as they leverage…

Data mining is the task of discovering interesting patterns from large amounts of data. There are many data mining tasks, such as classification, clustering, association rule mining, and sequential pattern mining. Sequential pattern mining…

The problem of joint sequential detection and isolation is considered in the context of multiple, not necessarily independent, data streams. A multiple testing framework is proposed, where each hypothesis corresponds to a different subset…

We introduce a class of stochastic integer sequences. In these sequences, every element is a sum of two previous elements, at least one of which is chosen randomly. The interplay between randomness and memory underlying these sequences…

We consider the first-order theory of random variables with the probabilistic independence relation, which concerns statements consisting of random variables, the probabilistic independence symbol, logical operators, and existential and…

Constrained sequential pattern mining aims at identifying frequent patterns on a sequential database of items while observing constraints defined over the item attributes. We introduce novel techniques for constraint-based sequential…

We study sequences of partitions of the unit interval into subintervals, starting from the trivial partition, in which each partition is obtained from the one before by splitting its subintervals in two, according to a given rule, and then…

A graph generative model defines a distribution over graphs. One type of generative model is constructed by autoregressive neural networks, which sequentially add nodes and edges to generate a graph. However, the likelihood of a graph under…

A general approach is established for deriving one-shot performance bounds for information-theoretic problems on general alphabets beyond countable alphabets. It is mainly based on the quantization idea and a novel form of "likelihood…

This paper considers estimation of a univariate density from an individual numerical sequence. It is assumed that (i) the limiting relative frequencies of the numerical sequence are governed by an unknown density, and (ii) there is a known…

The problem of distributed function computation is studied, where functions to be computed is not necessarily symbol-wise. A new method to derive a converse bound for distributed computing is proposed; from the structure of functions to be…

We study how macroscopic observational constraints restrict admissible microscopic explanatory structures when no intrinsic order or dynamics is assumed a priori. Starting from an unordered collection of measurement outcomes, we formulate…

Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…

Inference of causality is central in nonlinear time series analysis and science in general. A popular approach to infer causality between two processes is to measure the information flow between them in terms of transfer entropy. Using…