Related papers: A Projection Method for Derivation of Non-Shannon-…
We develop a systematic information-theoretic framework for quantification and mitigation of error in probabilistic Lagrangian (i.e., path-based) predictions which are obtained from dynamical systems generated by uncertain (Eulerian) vector…
Based on an apparently new Lagrange-type identity, a Cauchy--Schwarz-type inequality is proved. The mentioned identity is obtained by using certain ``macro'' variables; it is hoped that such a method can be used to prove or produce other…
One of the main unsolved problems of cosmology is how to maximize the extraction of information from nonlinear data. If the data are nonlinear the usual approach is to employ a sequence of statistics (N-point statistics, counting statistics…
Entanglement detection typically relies on linear inequalities for mean values of certain observables (entanglement witnesses), where violation indicates entanglement. We provide a general method to improve any of these inequalities for…
Conformal prediction is a popular uncertainty quantification method that augments a base predictor to return sets of predictions with statistically valid coverage guarantees. However, current methods are often computationally expensive and…
Information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems. Examples are Nonnegative Matrix/Tensor Factorization, Stochastic Neighbor…
Estimating the Shannon information associated with individual neurons is a non-trivial problem. Three key methods used to estimate the mutual information between neuron inputs and outputs are described, and a list of further readings is…
In an effort to develop the foundations for a non-stochastic theory of information, the notion of $\delta$-mutual information between uncertain variables is introduced as a generalization of Nair's non-stochastic information functional.…
By revisiting the mathematical foundation of the uncertainty relation, skew information-based uncertainty sequences are developed for any two quantum channels. A reinforced version of the Cauchy-Schwarz inequality is adopted to improve the…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
We provide identification results for a broad class of learning models in which continuous outcomes depend on three types of unobservables: known heterogeneity, initially unknown heterogeneity that may be revealed over time, and transitory…
In 1975 James Pickands III showed that the excesses over a high threshold are approximatly Generalized Pareto distributed. Since then, a variety of estimators for the parameters of this cdf have been studied, but always assuming the…
In this paper, we first present simple proofs of Choi's results [4], then we give a short alternative proof for Fiedler and Markham's inequality [6]. We also obtain additional matrix inequalities related to partial determinants.
This work develops problem statements related to encoders and autoencoders with the goal of elucidating variational formulations and establishing clear connections to information-theoretic concepts. Specifically, four problems with varying…
It is increasingly common to encounter time-varying random fields on networks (metabolic networks, sensor arrays, distributed computing, etc.). This paper considers the problem of optimal, nonlinear prediction of these fields, showing from…
Several variations of the Heisenberg uncertainty inequality are derived on the basis of "noise-resolution duality" recently proposed by the authors. The same approach leads to a related inequality that provides an upper limit for the…
Starting from the pioneering works of Shannon and Weiner in 1948, a plethora of works have been reported on entropy in different directions. Entropy-related review work in the direction of statistics, reliability and information science, to…
Synthetic indices are used in Economics to measure various aspects of monetary inequalities. These scalar indices take as input the distribution over a finite population, for example the population of a specific country. In this article we…
New sum and product uncertainty relations, containing variances of three or four observables, but not containing explicitly their covariances, are derived. One of consequences is the new inequality, giving a nonzero lower bound for the…
Traditional machine learning relies on explicit models and domain assumptions, limiting flexibility and interpretability. We introduce a model-free framework using surprisal (information theoretic uncertainty) to directly analyze and…