Related papers: $f$-Divergence Inequalities via Functional Dominat…
This is a survey paper with some original results of the author on refined versions of the Azuma-Hoeffding inequality with some examples that are related to information theory. This work has evolved to the joint paper with Maxim Raginsky in…
Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric…
Some inequalities for functions of bounded variation that provide reverses for the inequality between the integral mean and the p-norm are established. Applications related to the celebrated Landau inequality between the norms of the…
Some limit theorems of the type $\int_{\Omega}f_n dm_n -- --> \int_{\Omega}f dm$ are presented for scalar, (vector), (multi)-valued sequences of m_n-integrable functions f_n. The convergences obtained, in the vector and multivalued…
In this work, we introduce novel information-theoretic generalization bounds using the conditional $f$-information framework, an extension of the traditional conditional mutual information (MI) framework. We provide a generic approach to…
We generalize McDiarmid's inequality for functions with bounded differences on a high probability set, using an extension argument. Those functions concentrate around their conditional expectations. We further extend the results to…
We show that various functionals related to the supremum of a real function defined on an arbitrary set or a measure space are Hadamard directionally differentiable. We specifically consider the supremum norm, the supremum, the infimum, and…
In this paper, we propose a novel, effective and efficient probabilistic pruning criterion for probabilistic similarity queries on uncertain data. Our approach supports a general uncertainty model using continuous probabilistic density…
We state some inequalities for m-divisible and infinite divisible characteristic functions. Basing on them we propose a statistical test for a distribution to be infinitely divisible. Keywords: infinite divisible distributions; statistical…
We formulate a new information-theoretic principle--the shifted composition rule--which bounds the divergence (e.g., Kullback-Leibler or R\'enyi) between the laws of two stochastic processes via the introduction of auxiliary shifts. In this…
In this paper we introduce the concepts of p-logarithmic and Alpha-power divergence measures and point out a number of basic results.
The focal-loss has become a widely used alternative to cross-entropy in class-imbalanced classification problems, particularly in computer vision. Despite its empirical success, a systematic information-theoretic study of the focal-loss…
This paper is concerned with the study of constrained statistical learning problems, the unconstrained version of which are at the core of virtually all of modern information processing. Accounting for constraints, however, is paramount to…
Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only…
We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational…
A set $D$ of vertices in a graph $G=(V,E)$ is a degree restricted dominating set for $G$ if each vertex $v_i$ in $D$ is dominating atmost $g(d_i)$ vertices of $V-D$, where $g$ is a function restricting the degree value $d_i$ with respect to…
We extend the classical Lebesgue and Fubini differentiation theorems to functions of several variables, using the notions of joint derivative and joint monotonicity. Our first main result shows that for a function $f$ of bounded variation,…
TxGraffiti is an automated conjecturing program that produces graph theoretic conjectures in the form of conjectured inequalities. This program written and maintained by the second author since 2017 was inspired by the successes of previous…
Class imbalance poses a significant challenge in classification tasks, where traditional approaches often lead to biased models and unreliable predictions. Undersampling and oversampling techniques have been commonly employed to address…
Basic principles of set theory have been applied in the context of probability and binary computation. Applying the same principles on inequalities is less common but can be extremely beneficial in a variety of fields. This paper formulates…