相关论文: Sup-sums principles for F-divergence, Kullback--Le…
Integration at a point is a new kind of integration derived from integration over an interval in infinitesimal and infinity domains which are spaces larger than the reals. Consider a continuous monotonic divergent function that is…
As a continuation of the paper [20] on standard $f$-divergences, we make a systematic study of maximal $f$-divergences in general von Neumann algebras. For maximal $f$-divergences, apart from their definition based on Haagerup's…
We consider the problem of estimating probability density functions based on sample data, using a finite mixture of densities from some component class. To this end, we introduce the $h$-lifted Kullback--Leibler (KL) divergence as a…
We consider Bayesian linear inverse problems in infinite-dimensional separable Hilbert spaces, with a Gaussian prior measure and additive Gaussian noise model, and provide an extension of the concept of Bayesian D-optimality to the…
We derive an optimal bound on the sum of entropic uncertainties of two or more observables when they are sequentially measured on the same ensemble of systems. This optimal bound is shown to be greater than or equal to the bounds derived in…
Most quantum divergences derive their structure from classical f-divergences or Renyi-type constructions, a dependence that obscures several quantum geometric effects. We introduce a quantum relative-alpha-entropy that extends Umegaki's…
In this note, with the help of the boundary classification of diffusions, we derive a criterion of the convergence of perpetual integral functionals of transient real-valued diffusions. In the particular case of transient Bessel processes,…
New upper bounds on the relative entropy are derived as a function of the total variation distance. One bound refines an inequality by Verd\'{u} for general probability measures. A second bound improves the tightness of an inequality by…
We present new counterexamples, which provide stronger limitations to sums-differences statements than were previously known. The main idea is to consider non-uniform probability measures.
The Fokker-Planck equation can be reformulated as a continuity equation, which naturally suggests using the associated velocity field in particle flow methods. While the resulting probability flow ODE offers appealing properties - such as…
We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality,…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
Entropy-like functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most well-known are the von Neumann entropy $trace (\rho\log \rho)$ and a generalization of the…
The logistic loss (a.k.a. cross-entropy loss) is one of the most popular loss functions used for multiclass classification. It is also the loss function of choice for next-token prediction in language modeling. It is associated with the…
Good robust estimators can be tuned to combine a high breakdown point and a specified asymptotic efficiency at a central model. This happens in regression with MM- and tau-estimators among others. However, the finite-sample efficiency of…
Furstenberg has associated to every topological group $G$ a universal boundary $\partial(G)$. If we consider in addition a subgroup $H<G$, the relative notion of $(G,H)$-boundaries admits again a maximal object $\partial(G,H)$. In the case…
This paper extends some geometric properties of a one-parameter family of relative entropies. These arise as redundancies when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative…
$f$-divergences, which quantify discrepancy between probability distributions, are ubiquitous in information theory, machine learning, and statistics. While there are numerous methods for estimating $f$-divergences from data, a limit…
Let $(X,T)$ be a topological dynamical system. We define the measure-theoretical lower and upper entropies $\underline{h}_\mu(T)$, $\bar{h}_\mu(T)$ for any $\mu\in M(X)$, where $M(X)$ denotes the collection of all Borel probability measures…
We give conditions on a finitary endofunctor of a finitely accessible category to admit a final coalgebra. Our conditions always apply to the case of a finitary endofunctor of a locally finitely presentable (l.f.p.) category and they bring…