Related papers: Maximum information divergence from linear and tor…
We define a class of probability distributions that we call simplicial mixture models, inspired by simplicial complexes from algebraic topology. The parameters of these distributions represent their topology and we show that it is possible…
Inference capabilities of machine learning (ML) systems skyrocketed in recent years, now playing a pivotal role in various aspect of society. The goal in statistical learning is to use data to obtain simple algorithms for predicting a…
Many complex multi-target prediction problems that concern large target spaces are characterised by a need for efficient prediction strategies that avoid the computation of predictions for all targets explicitly. Examples of such problems…
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
In algebraic statistics, the maximum likelihood degree of a statistical model refers to the number of solutions (counted with multiplicity) of the score equations over the complex field. In this paper, the maximum likelihood degree of the…
Information bottleneck (IB) is a paradigm to extract information in one target random variable from another relevant random variable, which has aroused great interest due to its potential to explain deep neural networks in terms of…
We view the Information Bottleneck Principle (IBP: Tishby et al., 1999; Schwartz-Ziv and Tishby, 2017) and Predictive Information Bottleneck Principle (PIBP: Still et al., 2007; Alemi, 2019) as special cases of a family of general…
The maximum entropy principle is a powerful tool for solving underdetermined inverse problems. This paper considers the problem of discretizing a continuous distribution, which arises in various applied fields. We obtain the approximating…
Entropic regularization is a method for large-scale linear programming. Geometrically, one traces intersections of the feasible polytope with scaled toric varieties, starting at the Birch point. We compare this to log-barrier methods, with…
We introduce estimation and test procedures through divergence optimization for discrete or continuous parametric models. This approach is based on a new dual representation for divergences. We treat point estimation and tests for simple…
In this paper, we present a new numerical method for determining the numerical solution of interface problems to optimal accuracy with respect to the polynomial order of the Lagrangian finite element space on polytopial meshes. We introduce…
We consider the distance minimization problem to a real algebraic variety $X \subseteq \RR^n$ when the metric is induced by a polyhedral norm. Each point in the variety has a Voronoi cell whose geometry depends on the normal space at the…
We study statistical estimation under local differential privacy (LDP) when users may hold heterogeneous privacy levels and accuracy must be guaranteed with high probability. Departing from the common in-expectation analyses, and for…
We consider a strongly heterogeneous medium saturated by an incompressible viscous fluid as it appears in geomechanical modeling. This poroelasticity problem suffers from rapidly oscillating material parameters, which calls for a thorough…
Latent space models have been widely adopted in modeling network data. Developing statistical inference for estimated model parameters enables quantifying associated uncertainty and is pivotal for downstream tasks. Despite recent progress…
We study binary classification in the setting where the learner is presented with multiple corrupted training samples, with possibly different sample sizes and degrees of corruption, and introduce an approach based on minimizing a weighted…
The hypergeometric distributions have many important applications, but they have not had sufficient attention in information theory. Hypergeometric distributions can be approximated by binomial distributions or Poisson distributions. In…
We discuss averaged turbulence modeling of multi-scales of length for an incompressible Newtonian fluid, with the help of the maximum information principle. We suppose that there exists a function basis to decompose the turbulent…
The information bottleneck principle is an elegant and useful approach to representation learning. In this paper, we investigate the problem of representation learning in the context of reinforcement learning using the information…
This paper is devoted to the mathematical study of some divergences based on the mutual information well-suited to categorical random vectors. These divergences are generalizations of the "entropy distance" and "information distance". Their…