Related papers: Generalized Information Inequalities via Submodula…
We study probability inequalities leading to tail estimates in a general semigroup $\mathscr{G}$ with a translation-invariant metric $d_{\mathscr{G}}$. (An important and central example of this in the functional analysis literature is that…
In this paper, we study the classic submodular maximization problem subject to a group equality constraint under both non-adaptive and adaptive settings. It has been shown that the utility function of many machine learning applications,…
The Ingleton inequality is a classical linear information inequality that holds for representable matroids but fails to be universally valid for entropic vectors. Understanding the extent to which this inequality can be violated has been a…
Evaluating large language models across many benchmarks is expensive, yet many benchmarks are highly correlated. We formalize the selection of a small, informative subset as submodular maximization under a multivariate Gaussian model.…
Leveraging tools from convex analysis and incorporating additional singular value information of matrices, we completely resolve the problem of establishing perturbation bounds for the Frobenius norm of subunitary and positive polar…
Several matrix/operator inequalies are given. Most of them are unexpected extensions of the Araki Log-majorization theorem, obtained thanks to a new log-majorization for positive linear maps and normal operators (Theorem 2.9). The main idea…
In the nice recent work [48], S. Wang established uniform log-Sobolev inequalities for mean field particles when the energy is flat convex. In this note we comment how to extend his proof to some semi-convex energies provided the curvature…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
This paper presents a novel information-theoretic perspective on generalization in machine learning by framing the learning problem within the context of lossy compression and applying finite blocklength analysis. In our approach, the…
Using observation data to estimate unknown parameters in computational models is broadly important. This task is often challenging because solutions are non-unique due to the complexity of the model and limited observation data. However,…
Despite substantial progress in promoting fairness in high-stake applications using machine learning models, existing methods often modify the training process, such as through regularizers or other interventions, but lack formal guarantees…
Submodular optimization has become a fundamental paradigm for data selection, retrieval, summarization, and representation learning due to its ability to model coverage, diversity, and representativeness. However, classical submodular…
We present a generic functional inequality on Riemannian manifolds, both in additive and multiplicative forms, that produces well known and genuinely new Hardy-type inequalities. For the additive version, we introduce Riccati pairs that…
This paper is concerned with a generalized Halanay inequality and its applications to fractional-order delay linear systems. First, based on a sub-semigroup property of Mittag-Leffler functions, a generalized Halanay inequality is…
We present a new alternative theorems for sequences of functions. As applications, we extend recent results in the literature related to first-order necessary conditions for optimality problems. Our contributions involve extending…
There is a parallelism between Shannon information theory and algorithmic information theory. In particular, the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of…
Submodular functions describe a variety of discrete problems in machine learning, signal processing, and computer vision. However, minimizing submodular functions poses a number of algorithmic challenges. Recent work introduced an…
We introduce new classes of informational functionals, called \emph{upper moments}, respectively \emph{down-Fisher measures}, obtained by applying classical functionals such as $p$-moments and the Fisher information to the recently…
This work concerns the local convergence theory of Newton and quasi-Newton methods for convex-composite optimization: minimize f(x):=h(c(x)), where h is an infinite-valued proper convex function and c is C^2-smooth. We focus on the case…
We strengthen Han's Fourier entropy-influence inequality $$ H[\widehat{f}] \leq C_{1}I(f) + C_{2}\sum_{i\in [n]}I_{i}(f)\ln\frac{1}{I_{i}(f)} $$ originally proved for $\{-1,1\}$-valued Boolean functions with $C_{1}=3+2\ln 2$ and $C_{2}=1$.…