Related papers: Relative entropy convergence under Picard's iterat…
In this paper, we derive exponential ergodicity in relative entropy for general kinetic SDEs under a partially dissipative condition. It covers non-equilibrium situations where the forces are not of gradient type and the invariant measure…
We define a new divergence of von Neumann algebras using a variational expression that is similar in nature to Kosaki's formula for the relative entropy. Our divergence satisfies the usual desirable properties, upper bounds the sandwiched…
In this paper we derive an integral (with respect to time) representation of the relative entropy (or Kullback-Leibler Divergence) between measures mu and P on the space of continuous functions from time 0 to T. The underlying measure P is…
Defining a divergence between the laws of continuous martingales is a delicate task, owing to the fact that these laws tend to be singular to each other. An important idea, put forward by N. Gantert, is to instead consider a scaling limit…
We consider and analyze applying a spectral inverse iteration algorithm and its subspace iteration variant for computing eigenpairs of an elliptic operator with random coefficients. With these iterative algorithms the solution is sought…
Consider generalized adapted stochastic integrals with respect to independently scattered random measures with second moments. We use a decoupling technique, known as the "principle of conditioning", to study their stable convergence…
We formulate a stochastic description of entropy production in scattering theory for coherent transport. We distinguish between the information entropy change due to partial knowledge of the leads' state and the thermodynamic entropy change…
This work develops new results for stochastic approximation algorithms. The emphases are on treating algorithms and limits with discontinuities. The main ingredients include the use of differential inclusions, set-valued analysis, and…
In [1] it is shown that recurrent neural networks (RNNs) can learn - in a metric entropy optimal manner - discrete time, linear time-invariant (LTI) systems. This is effected by comparing the number of bits needed to encode the…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the method is not new, it has not been extensively analyzed in…
Estimating mutual information from observed samples is a basic primitive, useful in several machine learning tasks including correlation mining, information bottleneck clustering, learning a Chow-Liu tree, and conditional independence…
This document expands upon the relationship between discrete and continuous entropy given in (Phys. Rev. Lett. 110 130407), \Violating Continuous Variable Einstein-Podolsky-Rosen Steering with Discrete Measurements". We provide a detailed…
The minimum rate needed to accurately approximate a product distribution based on an unnormalized informational divergence is shown to be a mutual information. This result subsumes results of Wyner on common information and Han-Verd\'{u} on…
We study the asymptotic stability of the semi-discrete (SD) numerical method for the approximation of stochastic differential equations. Recently, we examined the order of $\mathcal L^2$-convergence of the truncated SD method and showed…
We consider systems of stochastic evolutionary equations of the $p$-Laplace type. We establish convergence rates for a finite-element based space-time approximation, where the error is measured in a suitable quasi-norm. Under natural…
Following the theory of information measures based on the cumulative distribution function, we propose the fractional generalized cumulative entropy, and its dynamic version. These entropies are particularly suitable to deal with…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of…
In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of…