Related papers: Weighted information and entropy rates
One of the most important challenges in network science is to quantify the information encoded in complex network structures. Disentangling randomness from organizational principles is even more demanding when networks have a multiplex…
We combine an axiomatics of R\'{e}nyi with the $q$--deformed version of Khinchin axioms to obtain a measure of information (i.e., entropy) which accounts both for systems with embedded self-similarity and non-extensivity. We show that the…
Information functionals allow to quantify the degree of randomness of a given probability distribution, either absolutely (through min/max entropy principles) or relative to a prescribed reference one. Our primary aim is to analyze the…
The weight space of the Ising perceptron in which a set of random patterns is stored is examined using the generating function of the partition function $\phi(n)=(1/N)\log [Z^n]$ as the dimension of the weight vector $N$ tends to infinity,…
Stimulated by the need of describing useful notions related to information measures, we introduce the `pdf-related distributions'. These are defined in terms of transformation of absolutely continuous random variables through their own…
This paper addresses the problem of formalizing and quantifying the concept of "intensional inheritance" between two concepts. We begin by conceiving the intensional inheritance of $W$ from $F$ as the amount of information the proposition…
In this work the information loss in deterministic, memoryless systems is investigated by evaluating the conditional entropy of the input random variable given the output random variable. It is shown that for a large class of systems the…
We introduce and study the cumulative information generating function, which provides a unifying mathematical tool suitable to deal with classical and fractional entropies based on the cumulative distribution function and on the survival…
We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if,…
We introduce an information theoretic measure of statistical structure, called 'binding information', for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.'s…
The combinatorial basis of entropy by Boltzmann can be written $H= {N}^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy of a system, per unit entity, $N$ is the number of entities and $\mathbb{W}$ is the number of ways in which…
Shannon entropy and Fisher information functionals are known to quantify certain information-theoretic properties of continuous probability distributions of various origins. We carry out a systematic study of these functionals, while…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
Random matrix ensembles (RME) of quantum statistical Hamiltonian operators, {\em e.g.} Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), found applications in literature in study of following quantum…
A scheme of evaluating an impact of a given scientific paper based on importance of papers quoting it is investigated. Introducing a weight of a given citation, dependent on the previous scientific achievements of the author of the citing…
It is shown that if (M,phi,alpha) is a W*-dynamical system with M a type I von Neumann algebra then the entropy of alpha w.r.t. phi equals the entropy of the restriction of alpha to the center of M. If furthermore (N,psi,beta) is a…
In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
We consider a Bayesian problem of estimating of probability of success in a series of conditionally independent trials with binary outcomes. We study the asymptotic behaviour of differential entropy for posterior probability density…
For a wide range of functions $W\colon\mathbb{N}\to\mathbb{N}$, we establish a general result for estimating weighted averages of the form \[ \mathbb{E}^{W}_{n \le N} f(\vartheta(n))= \frac{1}{W(N)}\sum_{n=1}^N (W(n)-W(n-1))f(\vartheta(n)),…