Related papers: Minimax R\'enyi Redundancy
Consider a sequence $X^n$ of length $n$ emitted by a Discrete Memoryless Source (DMS) with unknown distribution $p_X$. The objective is to construct a lossless source code that maps $X^n$ to a sequence $\widehat{Y}^m$ of length $m$ that is…
The aim of this work is to provide bounds connecting two probability measures of the same event using R\'enyi $\alpha$-Divergences and Sibson's $\alpha$-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence…
This paper develops systematic approaches to obtain $f$-divergence inequalities, dealing with pairs of probability measures defined on arbitrary alphabets. Functional domination is one such approach, where special emphasis is placed on…
We consider the Minimum Description Length principle for online sequence prediction. If the underlying model class is discrete, then the total expected square loss is a particularly interesting performance measure: (a) this quantity is…
Computing problems that handle large amounts of data necessitate the use of lossless data compression for efficient storage and transmission. We present a novel lossless universal data compression algorithm that uses parallel computational…
This paper is concerned with the optimal number of redundant allocation to $n$-component coherent systems consist of heterogeneous dependent components. We assume that the system is built of $L$ groups of different components, $L\geq 1$,…
We study a proper definition of R\'enyi mutual information (RMI) in quantum field theory as defined via the Petz R\'enyi relative entropy. Unlike the standard definition, the RMI we compute is a genuine measure of correlations between…
Behavior Cloning (BC) is a widely adopted visual imitation learning method in robot manipulation. Current BC approaches often enhance generalization by leveraging large datasets and incorporating additional visual and textual modalities to…
The study of quantum and classical correlations between subsystems is fundamental to understanding many-body physics. In quantum information theory, the quantum mutual information, $I(A;B)$, is a measure of correlation between the…
Information theory has inspired numerous advancements in multi-view learning. Most multi-view methods incorporating information-theoretic principles rely an assumption called multi-view redundancy which states that common information…
The redundancy allocation problem is formulated minimizing the design cost for a series-parallel system with multiple component choices whereas ensuring a given system reliability level. The obtained model is a nonlinear integer programming…
In [Berta et al., J. Math. Phys. 56, 022205 (2015)], we recently proposed Renyi generalizations of the conditional quantum mutual information of a tripartite state on $ABC$ (with $C$ being the conditioning system), which were shown to…
Acyclic schemes posses known benefits for database design, speeding up queries, and reducing space requirements. An acyclic join dependency (AJD) is lossless with respect to a universal relation if joining the projections associated with…
The problem of detecting and removing redundant constraints is fundamental in optimization. We focus on the case of linear programs (LPs) in dictionary form, given by $n$ equality constraints in $n+d$ variables, where the variables are…
Deploying deep learning models, comprising of non-linear combination of millions, even billions, of parameters is challenging given the memory, power and compute constraints of the real world. This situation has led to research into model…
We consider the reversible exclusion process with reservoirs on arbitrary networks. We characterize the spectral gap, mixing time, and mixing window of the process, in terms of certain simple statistics of the underlying network. Among…
Recently, a new notion of quantum R\'enyi divergences has been introduced by M\"uller-Lennert, Dupuis, Szehr, Fehr and Tomamichel, J.Math.Phys. 54:122203, (2013), and Wilde, Winter, Yang, Commun.Math.Phys. 331:593--622, (2014), that has…
We investigate the cutoff phenomenon for Markov processes under information divergences such as $f$-divergences and R\'enyi divergences. We classify most common divergences into four types, namely $L^2$-type, $\mathrm{TV}$-type,…
We investigate the fundamental limits of reliable communication over multi-view channels, in which the channel output is comprised of a large number of independent noisy views of a transmitted symbol. We consider first the setting of…
We study the connection between the highly non-convex loss function of a simple model of the fully-connected feed-forward neural network and the Hamiltonian of the spherical spin-glass model under the assumptions of: i) variable…