Related papers: Entropy production and Kullback-Leibler divergence…
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic…
We present a new lower bound on the differential entropy rate of stationary processes whose sequences of probability density functions fulfill certain regularity conditions. This bound is obtained by showing that the gap between the…
We present a statistical mechanical framework based on the Kullback-Leibler divergence (KLD) to analyze the relativistic limits of decoding time-encoded information from a moving source. By modeling the symbol durations as…
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…
This paper presents an {\it ab initio} derivation of the expression given by irreversible thermodynamics for the rate of entropy production for different classes of diffusive processes. The first class are Lorentz gases, where…
The stochastic entropy generated during the evolution of a system interacting with an environment may be separated into three components, but only two of these have a non-negative mean. The third component of entropy production is…
This work presents a novel framework for time series analysis using entropic measures based on the kernel density estimate (KDE) of the time series' Takens' embeddings. Using this framework we introduce two distinct analytical tools: (1) a…
Recent years have seen a surge of interest in the algorithmic estimation of stochastic entropy production (EP) from trajectory data via machine learning. A crucial element of such algorithms is the identification of a loss function whose…
Recently, continual learning has received a lot of attention. One of the significant problems is the occurrence of \emph{concept drift}, which consists of changing probabilistic characteristics of the incoming data. In the case of the…
For a Markov process the detailed balance condition is equivalent to the time-reversibility of the process. For stochastic differential equations (SDE's) time discretization numerical schemes usually destroy the property of…
Entropy production quantifies the amount of irreversibility of a physical process, leading to fundamental bounds for thermodynamic quantities. It captures the inability to run a physical system forward and then backward, bringing it to the…
Parallel Kinetic Monte Carlo (KMC) is a potent tool to simulate stochastic particle systems efficiently. However, despite literature on quantifying domain decomposition errors of the particle system for this class of algorithms in the short…
The appealing theoretical measure of irreversibility in a stochastic process, as the ratio of the probabilities of a trajectory and its time reversal, cannot be accessed directly in experiment since the probability of a single trajectory is…
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These…
The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understand by examining the equation, an intuition and understanding…
The irreversible currents and entropy production rate of a dilute colloidal suspension are calculated using the linear irreversible thermodynamics and the linear response theory. The \anomalous" or \hidden" entropy recently discussed in the…
For deterministic continuous time nonlinear control systems, epsilon-practical stabilization entropy and practical stabilization entropy are introduced. Here the rate of attraction is specified by a KL-function. Upper and lower bounds for…
Infinitesimal contraction analysis provides exponential convergence rates between arbitrary pairs of trajectories of a system by studying the system's linearization. An essentially equivalent viewpoint arises through stability analysis of a…
Analyzing and controlling system entropy is a powerful tool for regulating predictability of control systems. Applications benefiting from such approaches range from reinforcement learning and data security to human-robot collaboration. In…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…