Related papers: A Projection Method for Derivation of Non-Shannon-…
Misclassification detection is an important problem in machine learning, as it allows for the identification of instances where the model's predictions are unreliable. However, conventional uncertainty measures such as Shannon entropy do…
Shannon Information theory has achieved great success in not only communication technology where it was originally developed for but also many other science and engineering fields such as machine learning and artificial intelligence.…
Many statistical models are given in the form of non-normalized densities with an intractable normalization constant. Since maximum likelihood estimation is computationally intensive for these models, several estimation methods have been…
This paper explores a new class of constrained difference programming problems, where the objective and constraints are formulated as differences of functions, without requiring their convexity. To investigate such problems, novel variants…
A number of methods have been introduced in order to measure the inequality in various situations such as income and expenditure. In order to curry out statistical inference, one often needs to estimate the available measures of inequality.…
In the contemporary era, the importance of information is undisputed, but there has never been a common understanding of information, nor a unanimous conclusion to the researches on information metrics. Based on the previous studies, this…
This book is not meant to be another compendium of select inequalities, nor does it claim to contain the latest or the slickest ways of proving them. This project is rather an attempt at describing how most functional inequalities are not…
For the first time, Schr\"odinger equations with cubic and more complex nonlinearities containing the unknown function with constant delay are analyzed. The physical considerations that can lead to the appearance of a delay in such…
Variance estimation is important for statistical inference. It becomes non-trivial when observations are masked by serial dependence structures and time-varying mean structures. Existing methods either ignore or sub-optimally handle these…
The presence of symmetries imposes a stringent set of constraints on a system. This constrained structure allows intelligent agents interacting with such a system to drastically improve the efficiency of learning and generalization, through…
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label…
It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…
A novel statistical approach based on the Wigner transform is proposed for the description of partially incoherent optical wave dynamics in nonlinear media. An evolution equation for the Wigner transform is derived from a nonlinear…
In this paper, some new forms of the Cheeger's inequalities are established for general (maybe unbounded) symmetric forms, the resulting estimates improve and extend the ones obtained by Lawler and Sokal (1988) for bounded jump processes.…
In a series of works, Lutwak, Yang and Zhang established what could be called affine information theory, which is the study of moment-entropy and Fisher-information-type inequalities that are invariant with respect to affine transformations…
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a…
There are three classical divergence measures in the literature on information theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson-Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic-geometric mean…
The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are…
The goal of this note is to show that some convolution type inequalities from Harmonic Analysis and Information Theory, such as Young's convolution inequality (with sharp constant), Nelson's hypercontractivity of the Hermite semi-group or…
This work addresses large dimensional covariance matrix estimation with unknown mean. The empirical covariance estimator fails when dimension and number of samples are proportional and tend to infinity, settings known as Kolmogorov…