Related papers: Threshold-free estimation of entropy from a Pearso…
In a sparse high-dimensional elliptical model we consider a hard threshold estimator for the correlation matrix based on Kendall's tau with threshold level $\alpha(\frac{\log p}{n})^{1/2}$. Parameters $\alpha$ are identified such that the…
Bayesian inference for Continuous-Time Markov Chains (CTMCs) on countably infinite spaces is notoriously difficult because evaluating the likelihood exactly is intractable. One way to address this challenge is to first build a non-negative…
In this paper, we present a novel method for computing the relative entropy as well as the expected relative entropy using an MCMC chain. The relative entropy from information theory can be used to quantify differences in posterior…
The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…
We present a sampling-free approach for computing the epistemic uncertainty of a neural network. Epistemic uncertainty is an important quantity for the deployment of deep neural networks in safety-critical applications, since it represents…
We consider the matrix completion problem where the aim is to esti-mate a large data matrix for which only a relatively small random subset of its entries is observed. Quite popular approaches to matrix completion problem are iterative…
It is well known that to estimate the Shannon entropy for symbolic sequences accurately requires a large number of samples. When some aspects of the data are known it is plausible to attempt to use this to more efficiently compute entropy.…
We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is…
We calculate the quantum Renyi entropy in a phase space representation for either fermions or bosons. This can also be used to calculate purity and fidelity, or the entanglement between two systems. We show that it is possible to calculate…
This paper studies decision-making and statistical inference for two-sided matching markets via matrix completion. In contrast to the independent sampling assumed in classical matrix completion literature, the observed entries, which arise…
Spectral methods are widely used to estimate eigenvectors of a low-rank signal matrix subject to noise. These methods use the leading eigenspace of an observed matrix to estimate this low-rank signal. Typically, the entrywise estimation…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
The information content of continuous quantum variables systems is usually studied using a number of well known approximation methods. The approximations are made to obtain the spectrum, eigenfunctions or the reduced density matrices that…
Tsallis entropy is a useful one-parameter generalization of the standard von Neumann entropy in information theory. We study the variance of Tsallis entropy of bipartite quantum systems in a random pure state. The main result is an exact…
Irreversibility is commonly quantified by entropy production. An external observer can estimate it through measuring an observable that is antisymmetric under time-reversal like a current. We introduce a general framework that, inter alia,…
Existing training criteria in automatic speech recognition(ASR) permit the model to freely explore more than one time alignments between the feature and label sequences. In this paper, we use entropy to measure a model's uncertainty, i.e.…
Conditional differential entropy provides an intuitive measure for relatively ranking time-series complexity by quantifying uncertainty in future observations given past context. However, its direct computation for high-dimensional…
Maximum Entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a…
We make use of recent results from random matrix theory to identify a derived threshold, for isolating noise from image features. The procedure assumes the existence of a set of noisy images, where denoising can be carried out on individual…
Complex scientific models where the likelihood cannot be evaluated present a challenge for statistical inference. Over the past two decades, a wide range of algorithms have been proposed for learning parameters in computationally feasible…