相关论文: Entropy and inference, revisited
Deep generative models that can tractably compute input likelihoods, including normalizing flows, often assign unexpectedly high likelihoods to out-of-distribution (OOD) inputs. We mitigate this likelihood paradox by manipulating input…
When deploying a trained machine learning model in the real world, it is inevitable to receive inputs from out-of-distribution (OOD) sources. For instance, in continual learning settings, it is common to encounter OOD samples due to the…
The question of how best to estimate a continuous probability density from finite data is an intriguing open problem at the interface of statistics and physics. Previous work has argued that this problem can be addressed in a natural way…
Penalized regression methods, such as $L_1$ regularization, are routinely used in high-dimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is…
We prove an entropy version of van der Corput's difference theorem: the entropy of a sequence is equal to the entropy of its differences. This reveals a potential correspondence between the theory of uniform distribution mod 1 and entropy.…
This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
A compound Poisson process whose parameters are all unknown is observed at finitely many equispaced times. Nonparametric estimators of the jump and L\'evy distributions are proposed and functional central limit theorems using the uniform…
Tight lower and upper bounds on the ratio of relative entropies of two probability distributions with respect to a common third one are established, where the three distributions are collinear in the standard $(n-1)$-simplex. These bounds…
We study properties of ridge functions $f(x)=g(a\cdot x)$ in high dimensions $d$ from the viewpoint of approximation theory. The considered function classes consist of ridge functions such that the profile $g$ is a member of a univariate…
In dealing with asymptotic approximation of possibly divergent nets of probability distributions, we are led to study uniform structures on the set of distributions. This paper identifies a class of such uniform structures that may be…
We show that for a $\mathbb{Z}^{l}$-action (or $(\N\cup\{0\})^l$-action) on a non-empty compact metrizable space $\Omega$, the existence of a affine space dense in the set of continuous functions on $\Omega$ constituted by elements…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
This paper explores methods for estimating or approximating the total variation distance and the chi-squared divergence of probability measures within topological sample spaces, using independent and identically distributed samples. Our…
Dirichlet process mixtures are particularly sensitive to the value of the precision parameter controlling the behavior of the latent partition. Randomization of the precision through a prior distribution is a common solution, which leads to…
Parameter estimation in diffusion processes from discrete observations up to a first-hitting time is clearly of practical relevance, but does not seem to have been studied so far. In neuroscience, many models for the membrane potential…
In this note a notion of generalized topological entropy for arbitrary subsets of the space of all sequences in a compact topological space is introduced. It is shown that for a continuous map on a compact space the generalized topological…
We consider the Dirichlet problem for elliptic systems with periodically distributed inclusions whose conduction parameter exhibits a significant contrast compared to the background media. We develop a unified method to quantify the…
The sequence of entropy numbers quantifies the degree of compactness of a linear operator acting between quasi-Banach spaces. We determine the asymptotic behavior of entropy numbers in the case of natural embeddings between…
In spaces of metrics, we investigate topological distributions of the doubling property, the uniform disconnectedness, and the uniform perfectness, which are the quasi-symmetrically invariant properties appearing in the David--Semmes…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…