Related papers: Differential entropy and time
We analyze a contrasting dynamical behavior of Gibbs-Shannon and conditional Kullback-Leibler entropies, induced by time-evolution of continuous probability distributions. The question of predominantly purpose-dependent entropy definition…
We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
We review the behaviour of the Gibbs' and conditional entropies in deterministic and stochastic systems and continue to a formulation appropriate for a stochastically perturbed system with delayed dynamics. The underlying question driving…
Entropic dynamics is a framework in which the laws of dynamics are derived as an application of entropic methods of inference. Its successes include the derivation of quantum mechanics and quantum field theory from probabilistic principles.…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
Shannon entropy and Fisher information functionals are known to quantify certain information-theoretic properties of continuous probability distributions of various origins. We carry out a systematic study of these functionals, while…
Quantum mechanics is derived as an application of the method of maximum entropy. No appeal is made to any underlying classical action principle whether deterministic or stochastic. Instead, the basic assumption is that in addition to the…
For generic systems exhibiting power law behaviors, and hence multiscale dependencies, we propose a new, and yet simple, tool to analyze multifractality and intermittency, after noticing that these concepts are directly related to the…
We study the temporal approach to equilibrium of the Gibbs' and conditional entropies for both invertible deterministic dynamics as well as non-invertible stochastic systems in the presence of white noise. The conditional entropy will…
For a dynamical system far from equilibrium, one has to deal with empirical probabilities defined through time-averages, and the main problem is then how to formulate an appropriate statistical thermodynamics. The common answer is that the…
The maximum entropy formalism developed by Jaynes determines the relevant ensemble in nonequilibrium statistical mechanics by maximising the entropy functional subject to the constraints imposed by the available information. We present an…
It is argued that a Gibbsian formula for the space-time distribution of microscopic trajectories of a nonequilibrium system provides a unifying framework for recent results on the fluctuations of the entropy production. The variable entropy…
It is well known that in Information Theory and Machine Learning the Kullback-Leibler divergence, which extends the concept of Shannon entropy, plays a fundamental role. Given an {\it a priori} probability kernel $\hat{\nu}$ and a…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
The Gibbs entropy of a macroscopic classical system is a function of a probability distribution over phase space, i.e., of an ensemble. In contrast, the Boltzmann entropy is a function on phase space, and is thus defined for an individual…
In macroscopic systems behavior is usually reproducible and fluctuations, which are deviations from the typically observed mean values, are small. But almost all inverse problems in the physical and biological sciences are ill-posed and…
Experimental designs are tools which can dramatically reduce the number of simulations required by time-consuming computer codes. Because we don't know the true relation between the response and inputs, designs should allow one to fit a…
We propose a variational framework for nonequilibrium thermodynamics built around the effective number of accessible state, a multiplicative count that ranges from for a uniform distribution to one under complete localization, and whose…
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…