Related papers: Discretizing Distributions with Exact Moments: Err…
We consider special upwinding Petrov-Galerkin discretizations for convection-diffusion problems. For the one dimensional case with a standard continuous linear element as the trial space and a special exponential bubble test space, we prove…
This is the second part of study on the optimal convergence rate of the explicit Euler discretization in time for the convection-diffusion equations [Appl. Math. Lett. \textbf{131} (2022) 108048] which focuses on high-dimensional…
The Kullback-Leibler (KL) divergence is frequently used in data science. For discrete distributions on large state spaces, approximations of probability vectors may result in a few small negative entries, rendering the KL divergence…
Strong approximation errors of both finite element semi-discretization and spatio-temporal full discretization are analyzed for the stochastic Allen-Cahn equation driven by additive noise in space dimension $d \leq 3$. The full…
We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the…
An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed. We first utilize the Cram\'er-Rao bound to identify the Gaussian setting as a fundamental…
In Bayesian statistics probability distributions express beliefs. However, for many problems the beliefs cannot be computed analytically and approximations of beliefs are needed. We seek a loss function that quantifies how "embarrassing" it…
We consider linear reaction-diffusion equations posed on unbounded domains, and discretized by adaptive Lagrange finite elements. To obtain finite-dimensional spaces, it is necessary to introduce a truncation boundary, whereby only a…
A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on…
The minimum rate needed to accurately approximate a product distribution based on an unnormalized informational divergence is shown to be a mutual information. This result subsumes results of Wyner on common information and Han-Verd\'{u} on…
The privacy loss distribution (PLD) provides a tight characterization of the privacy loss of a mechanism in the context of differential privacy (DP). Recent work has shown that PLD-based accounting allows for tighter $(\varepsilon,…
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine…
A rich line of recent work has studied distributionally robust learning approaches that seek to learn a hypothesis that performs well, in the worst-case, on many different distributions over a population. We argue that although the most…
We study Gaussian approximations to the distribution of a diffusion. The approximations are easy to compute: they are defined by two simple ordinary differential equations for the mean and the covariance. Time correlations can also be…
Recently, continual learning has received a lot of attention. One of the significant problems is the occurrence of \emph{concept drift}, which consists of changing probabilistic characteristics of the incoming data. In the case of the…
The state-of-the-art methods for estimating high-dimensional covariance matrices all shrink the eigenvalues of the sample covariance matrix towards a data-insensitive shrinkage target. The underlying shrinkage transformation is either…
Entropy based ideas find wide-ranging applications in finance for calibrating models of portfolio risk as well as options pricing. The abstracted problem, extensively studied in the literature, corresponds to finding a probability measure…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
We study the problem of discrete distribution estimation in KL divergence and provide concentration bounds for the Laplace estimator. We show that the deviation from mean scales as $\sqrt{k}/n$ when $n \ge k$, improving upon the best prior…
We consider a linear ill-posed equation in the Hilbert space setting. Multiple independent unbiased measurements of the right hand side are available. A natural approach is to take the average of the measurements as an approximation of the…