Related papers: Weighted information and entropy rates
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
Fisher information is a measure of the best precision with which a parameter can be estimated from statistical data. It can also be defined for a continuous random variable without reference to any parameters, in which case it has a…
It is important to properly correct for measurement error when estimating density functions associated with biomedical variables. These estimators that adjust for measurement error are broadly referred to as density deconvolution…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
The ratio between the probability that two distributions $R$ and $P$ give to points $x$ are known as importance weights or propensity scores and play a fundamental role in many different fields, most notably, statistics and machine…
The random matrix ensembles (RME), especially Gaussian random matrix ensembles GRME and Ginibre random matrix ensembles, are applied to following quantum systems: nuclear systems, molecular systems, and two-dimensional electron systems…
Filtering theory gives an explicit models for the flow of information and thereby quantifies the rates of change of information supplied to and dissipated from the filter's memory. Here we extend the analysis of Mitter and Newton from…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
The information-theoretic representation of quantum systems, which complements the familiar energy description of the density-functional and wave-function-based theories, is here discussed. According to it, the internal disorder of the…
We prove that the quantum relative entropy is a rate function in large deviation principle. Next, we define information criteria for quantum states and estimate the accuracy of the use of them. Most of the results in this paper are…
Systems which consist of many localized constituents interacting with each other can be represented by complex networks. Consistently, network science has become highly popular in vast fields focusing on natural, artificial and social…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
An algorithm to calculate the density of states, based on the well-known Wang-Landau method, is introduced. Independent random walks are performed in different restricted ranges of energy, and the resultant density of states is modified by…
Let $(X_i)_{i\geq 1}$ be an i.i.d. sample on $\RRR^d$ having density $f$. Given a real function $\phi$ on $\RRR^d$ with finite variation and given an integer valued sequence $(j_n)$, let $\fn$ denote the estimator of $f$ by wavelet…
We show a general phenomenon of the constrained functional value for densities satisfying general convexity conditions, which generalizes the observation in Bobkov and Madiman (2011) that the entropy per coordinate in a log-concave random…
Feng--Huang (2016) introduced weighted topological entropy and pressure for factor maps between dynamical systems and established its variational principle. Tsukamoto (2022) redefined those invariants quite differently for the simplest case…
The information in an individual finite object (like a binary string) is commonly measured by its Kolmogorov complexity. One can divide that information into two parts: the information accounting for the useful regularity present in the…
We discuss a new weighted likelihood method for parametric estimation. The method is motivated by the need for generating a simple estimation strategy which provides a robust solution that is simultaneously fully efficient when the model is…
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…