Related papers: Entropy of Closure Operators
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
Uniform continuity bounds on entropies are generally expressed in terms of a single distance measure between a pair of probability distributions or quantum states, typically, the total variation distance or trace distance. However, if an…
A basic systems question concerns the concept of closure, meaning autonomomy (closed) in the sense of describing the (sub)system as fully consistent within itself. Alternatively, the system may be nonautonomous (open) meaning it receives…
We determine the boundedness and compactness of a large class of operators, mapping from general Banach spaces of holomorphic functions into a particular type of spaces of functions determined by the growth of the functions, or the growth…
Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to…
We investigate the memory properties of discrete sequences built upon a finite number of states. We find that the block entropy can reliably determine the memory for systems modeled as Markov chains of arbitrary finite order. Further, we…
Characterising the capacity region for a network can be extremely difficult, especially when the sources are dependent. Most existing computable outer bounds are relaxations of the Linear Programming bound. One main challenge to extend…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
In this paper, we are mainly concerned with studying arbitrary unbounded square roots of linear operators as well as some of their basic properties. The paper contains many examples and counterexamples. As an illustration, we give explicit…
We prove an operator inequality that extends strong subadditivity of entropy: after taking a trace, the operator inequality becomes the strong subadditivity of entropy.
Neural networks achieve remarkable performance through superposition: encoding multiple features as overlapping directions in activation space rather than dedicating individual neurons to each feature. This challenges interpretability, yet…
The main theorem provides a characterisation of the finite rank operators lying in a norm closed Lie ideal of a continuous nest algebra. These operators are charaterised as those finite rank operators in the nest algebra satisfying a…
Parametrized families of density operators are studied. A generalization of the lower bound of Cramer and Rao is formulated. It involves escort density operators. The notion of phi-exponential family is introduced. This family, together…
The primary purpose of this paper is to investigate the question of invertibility of the sum of operators. The setting is bounded and unbounded linear operators. Some interesting examples and consequences are given. As an illustrative…
This paper studies the notion of computational entropy. Using techniques from convex optimization, we investigate the following problems: (a) Can we derandomize the computational entropy? More precisely, for the computational entropy, what…
In this paper, we consider contention resolution algorithms that are augmented with predictions about the network. We begin by studying the natural setup in which the algorithm is provided a distribution defined over the possible network…
The introduction of the categorical notion of closure operators has unified various important notions and has led to interesting examples and applications in diverse areas of mathematics (see for example, Dikranjan and Tholen (\cite{DT})).…
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
In this paper we obtain order estimates for entropy numbers of embeddings of weighted Sobolev spaces into weighted Lebesgue spaces and of weighted summation operators on trees. Here we consider some critical conditions on the parameters.
Data partitioning that maximizes/minimizes the Shannon entropy, or more generally the R\'enyi entropy is a crucial subroutine in data compression, columnar storage, and cardinality estimation algorithms. These partition algorithms can be…