Related papers: Empirical estimation of entropy functionals with c…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
A method of estimating the joint probability mass function of a pair of discrete random variables is described. This estimator is used to construct the conditional Shannon-R\'eyni-Tsallis entropies estimates. From there almost sure rates of…
In this work, we investigate the statistical computation of the Boltzmann entropy of statistical samples. For this purpose, we use both histogram and kernel function to estimate the probability density function of statistical samples. We…
Many scientific fields and applications require compact representations of multivariate functions. For this problem, decoupling methods are powerful techniques for representing the multivariate functions as a combination of linear…
We give a highly efficient "semi-agnostic" algorithm for learning univariate probability distributions that are well approximated by piecewise polynomial density functions. Let $p$ be an arbitrary distribution over an interval $I$ which is…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that…
We establish a sharp estimate for a minimal number of binary digits (bits) needed to represent all bounded total generalized variation functions taking values in a general totally bounded metric space $(E,\rho)$ up to an accuracy of…
Two estimates for the inverse binary entropy function are derived using the property of information entropy to estimate combinatorics of sequences as well as related formulas from population genetics for the effective number of alleles. The…
Consider a Poisson point process with unknown support boundary curve $g$, which forms a prototype of an irregular statistical model. We address the problem of estimating non-linear functionals of the form $\int \Phi(g(x))\,dx$. Following a…
The present paper considers a problem of estimating a linear functional $\Phi=\int_{-\infty}^\infty \varphi(x) f(x)dx$ of an unknown deconvolution density $f$ on the basis of i.i.d. observations $Y_i = \theta_i + \xi_i$ where $\xi_i$ has a…
We introduce an algorithm for estimating the entropy of pairwise, probabilistic graph models by leveraging bridges between social communities and an accurate entropy estimator on sparse samples. We propose using a measure of investment from…
Suppose that we wish to estimate a finite-dimensional summary of one or more function-valued features of an underlying data-generating mechanism under a nonparametric model. One approach to estimation is by plugging in flexible estimates of…
This paper discusses the amplitude estimation using data originating from a sine-like function as probability density function. If a simple least squares fit is used, a significant bias is observed for small amplitudes. It is shown that a…
A general framework of Numerical Singular Integrals (NSI) method based on the Integration By Parts (IBP) has been developed for integrals involving singular and nearly singular integrands, or NSI-IBP. Through a general integration by parts…
It is well known that the minimax rates of convergence of nonparametric density and regression function estimation of a random variable measured with error is much slower than the rate in the error free case. Surprisingly, we show that if…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
Trustworthiness in model predictions is crucial for safety-critical applications in the real world. However, deep neural networks often suffer from the issues of uncertainty estimation, such as miscalibration. In this study, we propose…
A new plug-in rule procedure for bandwidth selection in kernel circular density estimation is introduced. The performance of this proposal is checked throughout a simulation study considering a variety of circular distributions exhibiting…
Are score function estimators an underestimated approach to learning with $k$-subset sampling? Sampling $k$-subsets is a fundamental operation in many machine learning tasks that is not amenable to differentiable parametrization, impeding…