Related papers: Bayesian Entropy Estimation for Countable Discrete…
We study the problem of generating a random variate $X$ from a finite discrete probability distribution $P$ using an entropy source of independent fair coin flips. A classic result from Knuth and Yao shows that the optimal expected number…
Employing nonparametric methods for density estimation has become routine in Bayesian statistical practice. Models based on discrete nonparametric priors such as Dirichlet Process Mixture (DPM) models are very attractive choices due to…
Ewens-Pitman's partition structure arises as a system of sampling consistent probability distributions on set partitions induced by the Pitman-Yor process. It is widely used in statistical applications, particularly in species sampling…
In Bayesian inference for mixture models with an unknown number of components, a finite mixture model is usually employed that assumes prior distributions for mixing weights and the number of components. This model is called a mixture of…
It is shown that a simple Dirichlet process mixture of multivariate normals offers Bayesian density estimation with adaptive posterior convergence rates. Toward this, a novel sieve for non-parametric mixture densities is explored, and its…
The hierarchical Dirichlet process is a discrete random measure used as a prior in Bayesian nonparametrics and motivated by the study of groups of clustered data. We study the asymptotic behavior of the power sum symmetric polynomials for…
Entropy is a fundamental property of both classical and quantum systems, spanning myriad theoretical and practical applications in physics and computer science. We study the problem of obtaining estimates to within a multiplicative factor…
Bayesian hierarchical models are commonly employed for inference in count datasets, as they account for multiple levels of variation by incorporating prior distributions for parameters at different levels. Examples include Beta-Binomial,…
Dirichlet process mixture models (DPMM) play a central role in Bayesian nonparametrics, with applications throughout statistics and machine learning. DPMMs are generally used in clustering problems where the number of clusters is not known…
We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the…
An important functional of Poisson random measure is the negative binomial process (NBP). We use NBP to introduce a generalized Poisson-Kingman distribution and its corresponding random discrete probability measure. This random discrete…
We provide a general theorem bounding the error in the approximation of a random measure of interest--for example, the empirical population measure of types in a Wright-Fisher model--and a Dirichlet process, which is a measure having…
The study of almost surely discrete random probability measures is an active line of research in Bayesian nonparametrics. The idea of assuming interaction across the atoms of the random probability measure has recently spurred significant…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
Stick-breaking has a long history and is one of the most popular procedures for constructing random discrete distributions in Statistics and Machine Learning. In particular, due to their intuitive construction and computational tractability…
In this paper, we describe a Bayesian nonparametric approach to make inference for a bivariate spherically symmetric distribution. We consider a Dirichlet invariant process prior on the set of all bivariate spherically symmetric…
Let $X$ be a discrete random variable with support $S$ and $f : S \to S^\prime$ be a bijection. Then it is well-known that the entropy of $X$ is the same as the entropy of $f(X)$. This entropy preservation property has been well-utilized to…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
We provide a general method to analyze the asymptotic properties of a variety of estimators of continuous time diffusion processes when the data are not only discretely sampled in time but the time separating successive observations may…
Although discrete mixture modeling has formed the backbone of the literature on Bayesian density estimation, there are some well known disadvantages. We propose an alternative class of priors based on random nonlinear functions of a uniform…