相关论文: Cross Entropy Approximation of Structured Covarian…
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…
Classifier-guided diffusion models generate conditional samples by augmenting the reverse-time score with the gradient of the log-probability predicted by a probabilistic classifier. In practice, this classifier is usually obtained by…
We describe an approach to improving model fitting and model generalization that considers the entropy of distributions of modelling residuals. We use simple simulations to demonstrate the observational signatures of overfitting on ordered…
This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs…
A novel principle is presented which allows for the proof of bounded weak solutions to a class of physically relevant, strongly coupled parabolic systems exhibiting a formal gradient-flow structure. The main feature of these systems is that…
This article is devoted to the study of overlap measures of densities of two exponential populations. Various Overlapping Coefficients, namely: Matusita's measure $\rho$, Morisita's measure $\lambda$ and Weitzman's measure $\Delta$. A new…
Coarse-grained measurements offer a scalable alternative to full state tomography for characterizing complex quantum dynamics. We show that observational entropy (OE), an information-theoretic entropy defined directly from finite-resolution…
This paper deals with the estimation of the modes of an univariate mixture when the number of components is known and when the component density are well separated. We propose an algorithm based on the minimization of the "kp" criterion we…
We consider a family of prior probability distributions of particular interest, all being defined on the three-dimensional convex set of two-level quantum systems. Each distribution is, following recent work of Petz and Sudar, taken to be…
Kolmogorov complexity of a finite binary word reflects both algorithmic structure and the empirical distribution of symbols appearing in the word. Words with symbol frequencies far from one half have smaller combinatorial richness and…
In this paper, we present a machine learning approach for estimating the number of incident wavefronts in a direction of arrival scenario. In contrast to previous works, a multilayer neural network with a cross-entropy objective is trained.…
Existing rumor detection methods often neglect the content within images as well as the inherent relationships between contexts and images across different visual scales, thereby resulting in the loss of critical information pertinent to…
We study the problem of closeness testing for continuous distributions and its implications for causal discovery. Specifically, we analyze the sample complexity of distinguishing whether two multidimensional continuous distributions are…
This paper is concerned with non-parametric estimation of the entropy in ranked set sampling. Theoretical properties of the proposed estimator are studied. The proposed estimator is compared with the rival estimator in simple random…
In the context of statistical learning, the Information Bottleneck method seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description…
The Information Bottleneck method is a learning technique that seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description length, and…
Probabilistic embeddings have several advantages over deterministic embeddings as they map each data point to a distribution, which better describes the uncertainty and complexity of data. Many works focus on adjusting the distribution…
The minimum Kullback entropy principle (mKE) is a useful tool to estimate quantum states and operations from incomplete data and prior information. In general, the solution of a mKE problem is analytically challenging and an approximate…
Modern image retrieval systems increasingly rely on the use of deep neural networks to learn embedding spaces in which distance encodes the relevance between a given query and image. In this setting, existing approaches tend to emphasize…
Similarity-sensitive entropy measures the uncertainty of a probability law relative to a similarity kernel that encodes the distinguishability between states. We develop a measure-theoretic treatment covering both finite similarity matrices…