Related papers: A Very Efficient Scheme for Estimating Entropy of …
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
In Bayesian inference, we seek to compute information about random variables such as moments or quantiles on the basis of {available data} and prior information. When the distribution of random variables is {intractable}, Monte Carlo (MC)…
We study $\ell_p$ sampling and frequency moment estimation in a single-pass insertion-only data stream. For $p \in (0,2)$, we present a nearly space-optimal approximate $\ell_p$ sampler that uses $\widetilde{O}(\log n \log(1/\delta))$ bits…
The need to estimate a particular quantile of a distribution is an important problem which frequently arises in many computer vision and signal processing applications. For example, our work was motivated by the requirements of many…
Spreading processes on graphs arise in a host of application domains, from the study of online social networks to viral marketing to epidemiology. Various discrete-time probabilistic models for spreading processes have been proposed. These…
We prove a variety of new and refined uniform continuity bounds for entropies of both classical random variables on an infinite state space and of quantum states of infinite-dimensional systems. We obtain the first tight continuity estimate…
Algorithms for rare event complex systems simulations are proposed. Compressed Sensing (CS) has {\it revolutionized} our understanding of limits in signal recovery and has forced us to re-define Shannon-Nyquist sampling theorem for sparse…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
Over the last few years, machine learning unlocked previously infeasible features for compression, such as providing guarantees for users' privacy or tailoring compression to specific data statistics (e.g., satellite images or audio…
We consider a discrete time stochastic model with infinite variance and study the mean estimation problem as in Wang and Ramdas (2023). We refine the Catoni-type confidence sequence (abbr. CS) and use an idea of Bhatt et al. (2022) to…
Distributed algorithms for network science applications are of great importance due to today's large real-world networks. In such algorithms, a node is allowed only to have local interactions with its immediate neighbors. This is because…
Estimation of the initial state of turbulent channel flow from limited data is investigated using an adjoint-variational approach. The data are generated from a reference direct numerical simulation (DNS) which is sub-sampled at different…
Low-rank approximation in data streams is a fundamental and significant task in computing science, machine learning and statistics. Multiple streaming algorithms have emerged over years and most of them are inspired by randomized…
Ever since Claude Shannon used entropy for his "Mathematical Theory of Communication", entropy has become a buzzword in research circles with scientists applying entropy to describe any phenomena that are reminiscent of disorder. In this…
The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically,…
This paper presents a tutorial for CS applications in communications networks. The Shannon's sampling theorem states that to recover a signal, the sampling rate must be as least the Nyquist rate. Compressed sensing (CS) is based on the…
We present two theorems demonstrating non-perturbatively the decrease under relevant renormalization group (RG) flow of two quantities, $c_{\text{eff}}$ and $g_{\text{eff}}$ characterizing, respectively, the universal information content of…
The recent extension of permutation entropy and its derivatives to graph signals has opened up new horizons for the analysis of complex, high-dimensional systems evolving on networks. However, these measures are all fundamentally rooted in…
The Tsallis and R\'enyi entropies are important quantities in the information theory, statistics and related fields because the Tsallis entropy is an one parameter generalization of the Shannon entropy and the R\'enyi entropy includes…