Related papers: Maximizing the Bregman divergence from a Bregman f…
We consider non-reversible perturbations of reversible diffusions that do not alter the invariant distribution and we ask whether there exists an optimal perturbation such that the rate of convergence to equilibrium is maximized. We solve…
We employ optimal control theory to study the problem of estimating the probability density function from a data set originating from an unknown probability distribution. The original variational problem is reformulated as a multi-stage…
Neural networks are popular state-of-the-art models for many different tasks.They are often trained via back-propagation to find a value of the weights that correctly predicts the observed data. Although back-propagation has shown good…
I consider a stochastic optimization problem for a time-changed Bessel process whose diffusion rate is constrained to be between two positive values $r_{1}<r_{2}$. The problem is to find an optimal adapted strategy for the choice of…
Divergences are fundamental to the information criteria that underpin most signal processing algorithms. The alpha-beta family of divergences, designed for non-negative data, offers a versatile framework that parameterizes and continuously…
Many inference problems involving questions of optimality ask for the maximum or the minimum of a finite set of unknown quantities. This technical report derives the first two posterior moments of the maximum of two correlated Gaussian…
The Kullback--Leibler divergence together with exponential families establishes the foundation of information geometry and is widely generalized. Among the generalization, we focus on the $(h,\tau)$-divergence and $(h,\tau)$-exponential…
The paper describes a generalized iterative proportional fitting procedure which can be used for maximum likelihood estimation in a special class of the general log-linear model. The models in this class, called relational, apply to…
The families of $f$-divergences (e.g. the Kullback-Leibler divergence) and Integral Probability Metrics (e.g. total variation distance or maximum mean discrepancies) are widely used to quantify the similarity between probability…
Neural networks allow us to model complex relationships between variables. We show how to efficiently find extrema of a trained neural network in regression problems. Finding the extremizing input of an approximated model is formulated as…
This paper focuses on the Bregman divergence defined by the reciprocal function, called the inverse divergence. For the loss function defined by the monotonically increasing function $f$ and inverse divergence, the conditions for the…
The problem of maximizing (or minimizing) the agreement between clusterings, subject to given marginals, can be formally posed under a common framework for several agreement measures. Until now, it was possible to find its solution only…
In this article we consider the problem of choosing an optimal sampling scheme for the regression problem simultaneously with that of model selection. We consider a batch type approach and an on-line approach following algorithms recently…
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased"…
Extracting relevant information from data is crucial for all forms of learning. The information bottleneck (IB) method formalizes this, offering a mathematically precise and conceptually appealing framework for understanding learning…
This survey presents the main results achieved for the influence maximization problem in social networks. This problem is well studied in the literature and, thanks to its recent applications, some of which currently deployed on the field,…
We consider the segmentation problem of univariate distributions from the exponential family with multiple parameters. In segmentation, the choice of the number of segments remains a difficult issue due to the discrete nature of the…
Selecting an optimal subset of features or instances under an information theoretic criterion has become an effective preprocessing strategy for reducing data complexity while preserving essential information. This study investigates two…
The aim of this short note is to present a solution to the discrete time exponential utility maximization problem in a case where the underlying asset has a multivariate normal distribution. In addition to the usual setting considered in…
The maximum likelihood degree of a statistical model refers to the number of solutions, where the derivative of the log-likelihood function is zero, over the complex field. This paper examines the maximum likelihood degree of the parameter…