Related papers: Directed Information on Abstract Spaces: Propertie…
Coherent wave propagation in disordered media gives rise to many fascinating phenomena as diverse as universal conductance fluctuations in mesoscopic metals and speckle patterns in light scattering. Here, the theory of electromagnetic wave…
Model compression and acceleration are attracting increasing attentions due to the demand for embedded devices and mobile applications. Research on efficient convolutional neural networks (CNNs) aims at removing feature redundancy by…
We consider discrete stochastic processes, modeled by classical master equations, on networks. The temporal growth of the lack of information about the system is captured by its non-equilibrium entropy, defined via the transition…
Distributionally robust optimization tackles out-of-sample issues like overfitting and distribution shifts by adopting an adversarial approach over a range of possible data distributions, known as the ambiguity set. To balance conservatism…
This paper builds upon our recent work which computed the moment generating function of the MIMO mutual information exactly in terms of a Painlev\'{e} V differential equation. By exploiting this key analytical tool, we provide an in-depth…
We develop an information-theoretic perspective on some questions in convex geometry, providing for instance a new equipartition property for log-concave probability measures, some Gaussian comparison results for log-concave measures, an…
In this paper, we consider the discrete memoryless interference channel with common information, in which two senders need deliver not only private messages but also certain common messages to their corresponding receivers. We derive an…
Without having direct access to the information that is being exchanged, traces of information flow can be obtained by looking at temporal sequences of user interactions. These sequences can be represented as causality trees whose…
High-dimensional representations for words, text, images, knowledge graphs and other structured data are commonly used in different paradigms of machine learning and data mining. These representations have different degrees of…
We study an information design problem with continuous state and discrete signal space. Under convex and S-shaped value functions, the optimal information structure is interval-partitional and exhibits a dual expectations property: each…
This paper considers distributed online nonconvex optimization with time-varying inequality constraints over a network of agents. For a time-varying graph, we propose a distributed online primal-dual algorithm with compressed communication…
In ordinal symbolic dynamics, transcripts describe the algebraic relationship between ordinal patterns. Using the concept of transcript, we exploit the mathematical structure of the group of permutations to derive properties and relations…
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
We define a notion of information that an individual sample provides to the training of a neural network, and we specialize it to measure both how much a sample informs the final weights and how much it informs the function computed by the…
One of the most fundamental questions one can ask about a pair of random variables X and Y is the value of their mutual information. Unfortunately, this task is often stymied by the extremely large dimension of the variables. We might hope…
This survey is concerned with the power of random information for approximation in the (deterministic) worst-case setting, with special emphasis on information consisting of functionals selected independently and identically distributed…
We provide tools to analyze information design problems subject to constraints. We do so by extending the insight in Le Treust and Tomala (2019) to the case of multiple inequality and equality constraints. Namely, that an information design…
We propose a distributed version of the Alternating Direction Method of Multipliers (ADMM) with linear updates for directed networks. We show that if the objective function of the minimization problem is smooth and strongly convex, our…
A partial differential equation governing the global evolution of the joint probability distribution of an arbitrary number of local flow observations, drawn randomly from a control volume, is derived and applied to examples involving…
This paper investigates the distributed fixed point seeking problem of sum-separable stochastic operators over the multi-agent network. Based on inexact Krasnosel'ski\u{\i}--Mann iterations, the communication-efficient distributed algorithm…