Related papers: Minimization Problems Based on Relative $\alpha$-E…
The concept of distinguishability lies at the heart of quantum information theory. We introduce \textit{left-right relative entropy} as a quantitative measure of distinguishability within the space of boundary states in two-dimensional…
In many problems in data mining and machine learning, data items that need to be clustered or classified are not points in a high-dimensional space, but are distributions (points on a high dimensional simplex). For distributions, natural…
This paper focuses on $\alpha$-divergence minimisation methods for Variational Inference. More precisely, we are interested in algorithms optimising the mixture weights of any given mixture model, without any information on the underlying…
The focal-loss has become a widely used alternative to cross-entropy in class-imbalanced classification problems, particularly in computer vision. Despite its empirical success, a systematic information-theoretic study of the focal-loss…
We obtain formulas for Petz-R\'enyi and Umegaki relative entropy from the idea of distribution of a positive selfadjoint operator. Classical results on R\'enyi and Kullback-Leibler divergences are applied to obtain new results and new…
Following [21, 23], the present work investigates a new relative entropy-regularized algorithm for solving the optimal transport on a graph problem within the randomized shortest paths formalism. More precisely, a unit flow is injected into…
We demonstrate that dual entropy expressions of the Tsallis type apply naturally to statistical-mechanical systems that experience an exceptional contraction of their configuration space. The entropic index $\alpha>1$ describes the…
Tsallis relative operator entropy was defined as a parametric extension of relative operator entropy and the generalized Shannon inequalities were shown in the previous paper. After the review of some fundamental properties of Tsallis…
The relative entropy of two n-party quantum states is an important quantity exhibiting, for example, the extent to which the two states are different. The relative entropy of the states formed by reducing two n-party to a smaller number $m$…
The effect of relative entropy asymmetry is analyzed in the context of empirical risk minimization (ERM) with relative entropy regularization (ERM-RER). Two regularizations are considered: $(a)$ the relative entropy of the measure to be…
The maximum entropy principle is a powerful tool for solving underdetermined inverse problems. This paper considers the problem of discretizing a continuous distribution, which arises in various applied fields. We obtain the approximating…
The effect of the relative entropy asymmetry is analyzed in the empirical risk minimization with relative entropy regularization (ERM-RER) problem. A novel regularization is introduced, coined Type-II regularization, that allows for…
The relative entropy between quantum states quantifies their distinguishability. The estimation of certain relative entropies has been investigated in the literature, e.g., the von Neumann relative entropy and sandwiched R\'enyi relative…
We give the tight bounds of Tsallis relative operator entropy by using Hermite-Hadamard's inequality. Some reverse inequalities related to Young inequalities are also given. In addition, operator inequalities for normalized positive linear…
Consider a defined density on a set of very large dimension. It is quite difficult to find an estimate of this density from a data set. However, it is possible through a projection pursuit methodology to solve this problem. Touboul's…
The equilibrium distributions of probabilities providing maximality of Renyi and Tsallis entropies are rederived. New S-forms of them are found which are normalised with corresponding entropies in contrast to the usual Z-forms normalised…
We revisit entropic formulations of the uncertainty principle for an arbitrary pair of positive operator-valued measures (POVM) $A$ and $B$, acting on finite dimensional Hilbert space. Salicr\'u generalized $(h,\phi)$-entropies, including…
The limits of scaled relative entropies between probability distributions associated with N-particle weakly interacting Markov processes are considered. The convergence of such scaled relative entropies is established in various settings.…
For linear inverse problems with Gaussian priors and Gaussian observation noise, the posterior is Gaussian, with mean and covariance determined by the conditioning formula. Using the Feldman-Hajek theorem, we analyse the prior-to-posterior…
Minimum divergence problems under integral constraints appear throughout statistics and probability, including sequential inference, bandit theory, and distributionally robust optimization. In many such settings, dual representations are…