相关论文: An identity of Chernoff bounds with an interpretat…

Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…

In this paper, we first introduce and define several new information divergences in the space of transition matrices of finite Markov chains which measure the discrepancy between two Markov chains. These divergences offer natural…

The paper deals with the order statistics and empirical mathematical expectation (which is also called the estimate of mathematical expectation in the literature) in the case of infinitely increasing random variables. The Kolmogorov concept…

Chernoff approximations to strongly continuous one-parameter semigroups give solutions to a wide class of differential equations. This paper studies the rate of convergence of the Chernoff approximations. We provide simple natural examples…

This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…

The method of optimizing entropy is used to (i) conduct Asymptotic Hypothesis Testing and (ii) determine the particle distribution for which Entropy is maximized. This paper focuses on two related applications of Information Theory:…

(abridged) In this paper, we present the issues we consider as essential as far as the statistical mechanics of finite systems is concerned. In particular, we emphasis our present understanding of phase transitions in the framework of…

Shannon entropy is the most crucial foundation of Information Theory, which has been proven to be effective in many fields such as communications. Renyi entropy and Chernoff information are other two popular measures of information with…

A unification of thermodynamics and information theory is proposed. It is argued that similarly to the randomness due to collisions in thermal systems, the quenched randomness that exists in data files in informatics systems contributes to…

We prove a lower bound on the relative entropy between two finite-dimensional states in terms of their entropy difference and the dimension of the underlying space. The inequality is tight in the sense that equality can be attained for any…

Two identities in statistical mechanics involving entropy differences (or ratios of density of states) at constant energy are derived. The first provides a nontrivial extension of the Jarzynski equality to the microcanonical ensemble [C.…

We develop a statistical mechanical interpretation of algorithmic information theory by introducing the notion of thermodynamic quantities, such as free energy, energy, statistical mechanical entropy, and specific heat, into algorithmic…

Chernoff's bound binds a tail probability (ie. $Pr(X \ge a)$, where $a \ge EX$). Assuming that the distribution of $X$ is $Q$, the logarithm of the bound is known to be equal to the value of relative entropy (or minus Kullback-Leibler…

The theory of large deviations deals with the probabilities of rare events (or fluctuations) that are exponentially small as a function of some parameter, e.g., the number of random components of a system, the time over which a stochastic…

The problem of the insensitivity of the macroscopic behavior of any thermodynamical system to partitioning generates a bias between the reproducibility of its macroscopic behavior viewed as the simplest form of causality and its long-term…

Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…

Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…

In this article we derive a useful expectation identity using the language of quantum statistical mechanics, where density matrices represent the state of knowledge about the system. This identity allows to establish relations between…

The Chernoff bound is an important inequality relation in probability theory. The original version of the Chernoff bound is to give an exponential decreasing bound on the tail distribution of sums of independent random variables. Recent…

In this paper we extend the results of Lenci and Rey-Bellet on the large deviation upper bound of the distribution measures of local Hamiltonians with respect to a Gibbs state, in the setting of translation-invariant finite-range…