Related papers: The Convex Information Bottleneck Lagrangian
Constrained optimization problems appear in a wide variety of challenging real-world problems, where constraints often capture the physics of the underlying system. Classic methods for solving these problems rely on iterative algorithms…
Lagrangian duality in mixed integer optimization is a useful framework for problems decomposition and for producing tight lower bounds to the optimal objective, but in contrast to the convex counterpart, it is generally unable to produce…
We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or…
This paper considers the problem of minimizing a convex expectation function with a set of inequality convex expectation constraints. We present a computable stochastic approximation type algorithm, namely the stochastic linearized proximal…
The immersed boundary (IB) method is an approach to fluid-structure interaction that uses Lagrangian variables to describe the structure and Eulerian variables to describe the fluid. Explicit time stepping schemes for the IB method require…
Deep neural networks (DNNs) have achieved significant success in various applications with large-scale and balanced data. However, data in real-world visual recognition are usually long-tailed, bringing challenges to efficient training and…
We propose new methods for optimizing the integration-by-parts (IBP) reduction of Feynman integrals, an important computational bottleneck in modern perturbative calculations in quantum field theory. Using the simple example of one-loop…
This paper addresses a class of general nonsmooth and nonconvex composite optimization problems subject to nonlinear equality constraints. We assume that a part of the objective function and the functional constraints exhibit local…
This paper presents an algorithmic study of a class of covering mixed-integer linear programming problems which encompasses classic cover problems, including multidimensional knapsack, facility location and supplier selection problems. We…
The optimal transport problem studies how to transport one measure to another in the most cost-effective way and has wide range of applications from economics to machine learning. In this paper, we introduce and study an information…
The continuous-time analysis of existing iterative algorithms for optimization has a long history. This work proposes a novel continuous-time control-theoretic framework for equality-constrained optimization. The key idea is to design a…
We study the computational complexity certification of inexact gradient augmented Lagrangian methods for solving convex optimization problems with complicated constraints. We solve the augmented Lagrangian dual problem that arises from the…
In this paper we analyze several inexact fast augmented Lagrangian methods for solving linearly constrained convex optimization problems. Mainly, our methods rely on the combination of excessive-gap-like smoothing technique developed in…
In this paper we address the problem of information-constrained optimal control for an interconnected system subject to one-step communication delays and power constraints. The goal is to minimize a finite-horizon quadratic cost by…
In this paper, we consider a class of convex programming problems with linear equality constraints, which finds broad applications in machine learning and signal processing. We propose a new adaptive balanced augmented Lagrangian (ABAL)…
Information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems. Examples are Nonnegative Matrix/Tensor Factorization, Stochastic Neighbor…
This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate…
In this work, we develop a control-theoretic framework for constrained optimization problems with composite objective functions including non-differentiable terms. Building on the proximal augmented Lagrangian formulation, we construct a…
We study the information bottleneck (IB) source coding problem, also known as remote lossy source coding under logarithmic loss. Based on a rate-limited description of noisy observations, the receiver produces a soft estimate for the remote…
Incremental learning aims to enable machine learning models to continuously acquire new knowledge given new classes, while maintaining the knowledge already learned for old classes. Saving a subset of training samples of previously seen…