相关论文: Characterizations and Kullback-Leibler Divergence …
This work sheds some light on the relationship between a distribution's standard deviation and its range, a topic that has been discussed extensively in the literature. While many previous studies have proposed inequalities or relationships…
In this paper, some new upper bounds for Kullback-Leibler divergence(KL-divergence) based on $L^1, L^2$ and $L^\infty$ norms of density functions are discussed. Our findings unveil that the convergence in KL-divergence sense sandwiches…
We study the problem of model selection type aggregation with respect to the Kullback-Leibler divergence for various probabilistic models. Rather than considering a convex combination of the initial estimators $f_1, \ldots, f_N$, our…
To characterize the Kullback-Leibler divergence and Fisher information in general parametrized hidden Markov models, in this paper, we first show that the log likelihood and its derivatives can be represented as an additive functional of a…
The Kullback-Leibler cluster entropy $\mathcal{D_{C}}[P \| Q] $ is evaluated for the empirical and model probability distributions $P$ and $Q$ of the clusters formed in the realized volatility time series of five assets (SP\&500, NASDAQ,…
We investigate the unbiased model for money exchanges: agents give at random time a dollar to one another (if they have one). Surprisingly, this dynamics eventually leads to a geometric distribution of wealth (shown empirically by…
The $\alpha$-divergences include the well-known Kullback-Leibler divergence, Hellinger distance and $\chi^2$-divergence. In this paper, we derive differential and integral relations between the $\alpha$-divergences that are generalizations…
Suppose Xt is either a regular exponential type Levy process or a Levy process with a bounded variation jumps measure. The distribution of the extrema of Xt play a crucial role in many financial and actuarial problems. This article employs…
Consider the nonparametric logistic regression problem. In the logistic regression, we usually consider the maximum likelihood estimator, and the excess risk is the expectation of the Kullback-Leibler (KL) divergence between the true and…
We prove characterization theorems for relative entropy (also known as Kullback-Leibler divergence), q-logarithmic entropy (also known as Tsallis entropy), and q-logarithmic relative entropy. All three have been characterized axiomatically…
In this article it is shown that in an equilibrium classical canonical ensemble of molecules with two-body interaction and external field full Gibbs distribution can be uniquely expressed in terms of a reduced two-particle distribution…
We derive a deterministic, non-asymptotic upper bound on the Kullback-Leibler (KL) divergence of the flow-matching distribution approximation. In particular, if the $L_2$ flow-matching loss is bounded by $\epsilon^2 > 0$, then the KL…
In this communication, the derivation of the Boltzmann-Gibbs and the Maxwellian distributions is presented from a geometrical point of view under the hypothesis of equiprobability. It is shown that both distributions can be obtained by…
A class of conserved models of wealth distributions are studied where wealth (or money) is assumed to be exchanged between a pair of agents in a population like the elastically colliding molecules of a gas exchanging energy. All sorts of…
We consider the problem of stochastic optimal control, where the state-feedback control policies take the form of a probability distribution and where a penalty on the entropy is added. By viewing the cost function as a Kullback- Leibler…
In this paper, we derive some upper and lower bounds and inequalities for the total variation distance (TVD) and the Kullback-Leibler divergence (KLD), also known as the relative entropy, between two probability measures $\mu$ and $\nu$…
In this paper we study a broad class of distribution functions which is defined by means of reflected generalized beta distribution. This class includes that of Beta-generated distribution as a special case. In particular, we use this class…
We characterize Martin-L\"of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus…
The envelope of an elliptical Gaussian complex vector, or equivalently, the amplitude or norm of a bivariate normal random vector has application in many weather and signal processing contexts. We explicitly characterize its distribution in…
Knowing if a model will generalize to data 'in the wild' is crucial for safe deployment. To this end, we study model disagreement notions that consider the full predictive distribution - specifically disagreement based on Hellinger…