Related papers: Source coding with escort distributions and Renyi …
Assessing whether two datasets are distributionally consistent is central to modern scientific analysis, particularly as generative artificial intelligence produces synthetic data whose fidelity must be validated against real observations…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Many axiomatic definitions of entropy, such as the R\'enyi entropy, of a random variable are closely related to the $\ell_{\alpha}$-norm of its probability distribution. This study considers probability distributions on finite sets, and…
Entropy and its various generalizations are important in many fields, including mathematical statistics, communication theory, physics and computer science, for characterizing the amount of information associated with a probability…
Recently, information theoretic analysis has become a popular framework for understanding the generalization behavior of deep neural networks. It allows a direct analysis for stochastic gradient/Langevin descent (SGD/SGLD) learning…
We study finite-length bounds for source coding with side information for Markov sources and channel coding for channels with conditional Markovian additive noise. For this purpose, we propose two criteria for finite-length bounds. One is…
For certain families of finite quantum graphs, we study the question of how eigenfunctions are distributed over the graph. To characterize properties of the distribution, generalized entropies of the R\'{e}nyi and Tsallis types are…
A novel lossless source coding paradigm applies to problems of unreliable lossless channels with low bit rates, in which a vital message needs to be transmitted prior to termination of communications. This paradigm can be applied to Alfred…
This paper investigates the general distributed lossless/lossy source coding formulated by Jana and Blahut. Their multi-letter rate-distortion region, an alternative to the region derived by Yang and Qin, is characterized by entropy…
We investigate dense coding by imposing various locality restrictions to our decoder by employing the resource theory of asymmetry framework. In this task, the sender Alice and the receiver Bob share an entangled state. She encodes the…
In this note, we studied the asymptotic behaviour of the length of the longest common substring for run-length encoded sequences. When the original sequences are generated by an $\alpha$-mixing process with exponential decay (or…
The Renyi, Shannon and Fisher spreading lengths of the classical or hypergeometric orthogonal polynomials, which are quantifiers of their distribution all over the orthogonality interval, are defined and investigated. These…
The distributed remote source coding (so-called CEO) problem is studied in the case where the underlying source, not necessarily Gaussian, has finite differential entropy and the observation noise is Gaussian. The main result is a new lower…
We show that the R\'enyi entropies of single particle, extended wave functions for disordered systems contain information about the multifractal spectrum. It is shown for moments of the R\'enyi entropy, $S_{n}$, where $|n|<1$, it is…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
A $q$-Gaussian measure is a generalization of a Gaussian measure. This generalization is obtained by replacing the exponential function with the power function of exponent $1/(1-q)$ ($q\neq 1$). The limit case $q=1$ recovers a Gaussian…
This paper presents a novel theoretical study of the general problem of multiple source adaptation using the notion of Renyi divergence. Our results build on our previous work [12], but significantly broaden the scope of that work in…
For a measurable function on a set which has a finite measure, an inequality holds between two Lp-norms. In this paper, we show similar inequalities for the Euclidean space and the Lebesgue measure by using a q-moment which is a moment of…
The paper suggests the concepts of an upper entropy and a lower entropy. We propose a new axiomatic definition, namely, upper entropy axioms, inspired by axioms of metric spaces, and also formulate lower entropy axioms. We also develop weak…