Related papers: A remark on conditional entropy
In this paper, we obtain fundamental $\mathcal{L}_{p}$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic relationships of the…
Irreversibility is commonly quantified by entropy production. An external observer can estimate it through measuring an observable that is antisymmetric under time-reversal like a current. We introduce a general framework that, inter alia,…
Microreversibility rules the fluctuations of the currents flowing across open systems in nonequilibrium (or equilibrium) steady states. As a consequence, the statistical cumulants of the currents and their response coefficients at arbitrary…
In a time-reversal non-invariant system the entropy production, dS/dt, may be negative. Close to the critical point of medium it is positive: larger or smaller as compared with one above critical point depending on whether the transitions…
We adapt the Kolmogorov-Sinai entropy to the non-extensive perspective recently advocated by Tsallis. The resulting expression is an average on the invariant distribution, which should be used to detect the genuine entropic index Q. We…
Comparison-based algorithms are algorithms for which the execution of each operation is solely based on the outcome of a series of comparisons between elements. Comparison-based computations can be naturally represented via the following…
The purpose of this work is to analyze the mathematical model governing motion of $n$-component, heat conducting reactive mixture of compressible gases. We prove sequential stability of weak variational entropy solutions when the state…
We consider a "length-biased" shift-dependent information measure, related to the differential entropy in which higher weight is assigned to large values of observed random variables. This allows us to introduce the notions of "weighted…
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio…
Here we define and study the properties of retrodictive inference. We derive equations relating retrodiction entropy and thermodynamic entropy, and as a special case, show that under equilibrium conditions, the two are identical. We…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…
Given an inverse problem with a normalizing flow prior, we wish to estimate the distribution of the underlying signal conditioned on the observations. We approach this problem as a task of conditional inference on the pre-trained…
As physics searches for invariants in observations, this paper looks for invariants of probabilistic observation without assuming physical structure. Structure emerges from the basic assumption of science that new information shall lead to…
Transfer entropy measures directed information flow in time series, and it has become a fundamental quantity in applications spanning neuroscience, finance, and complex systems analysis. However, existing estimation methods suffer from the…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
One of the crucial steps in scientific studies is to specify dependent relationships among factors in a system of interest. Given little knowledge of a system, can we characterize the underlying dependent relationships through observation…
Statistical physics aims to describe properties of macroscale systems in terms of distributions of their microscale agents. Its central tool is the maximization of entropy, a variational principle. We review the history of this principle,…
Irreversibility is a fundamental concept with important implications at many levels. It pinpoints the fundamental difference between the intrinsically reversible microscopic equations of motion and the unidirectional arrow of time that…
We prove the large deviation principle for several entropy and cross entropy estimators based on return times and waiting times on shift spaces over finite alphabets. We consider shift-invariant probability measures satisfying some…
In this paper, we introduce the notion of recurrence entropy in the context of nonlinear control systems. A set is said to be ($\tau$-)recurrent if every trajectory that starts in the set returns to it (within at most $\tau$ units of time).…