Related papers: Simplified Information Geometry Approach for Massi…
We introduce FedSGM, a unified framework for federated constrained optimization that addresses four major challenges in federated learning (FL): functional constraints, communication bottlenecks, local updates, and partial client…
We are concerned with two interrelated problems: smoothability of connection 1-forms with low regularity on bundles with prescribed smooth curvature 2-forms, and existence of isometric immersions with low regularity. We first show that if…
We construct an algorithm for implementing the discrete wavelet transform by means of matrices in SO_2(R) for orthonormal compactly supported wavelets and matrices in SL_m(R), m > = 2, for compactly supported biorthogonal wavelets. We show…
We study a distributed consensus-based stochastic gradient descent (SGD) algorithm and show that the rate of convergence involves the spectral properties of two matrices: the standard spectral gap of a weight matrix from the network…
We propose a generalized finite element method for the strongly damped wave equation with highly varying coefficients. The proposed method is based on the localized orthogonal decomposition introduced and is designed to handle independent…
Modern large-scale statistical models require to estimate thousands to millions of parameters. This is often accomplished by iterative algorithms such as gradient descent, projected gradient descent or their accelerated versions. What are…
This work focuses on the mixed membership models for multivariate categorical data widely used for analyzing survey responses and population genetics data. These grade of membership (GoM) models offer rich modeling power but present…
Natural policy gradient (NPG) is a common policy optimization algorithm and can be viewed as mirror ascent in the space of probabilities. Recently, Vaswani et al. [2021] introduced a policy gradient method that corresponds to mirror ascent…
Modern GPUs are equipped with large amounts of high-bandwidth memory, enabling them to support mini-batch sizes of up to tens of thousands of training samples. However, most existing optimizers struggle to perform effectively at such a…
An unavoidable task in quantum information processing is how to obtain data about the state of an individual system by suitable measurements. Informationally complete measurements are relevant in quantum state tomography, quantum…
We introduce Statistical Flow Matching (SFM), a novel and mathematically rigorous flow-matching framework on the manifold of parameterized probability measures inspired by the results from information geometry. We demonstrate the…
Conic optimization plays a crucial role in many machine learning (ML) problems. However, practical algorithms for conic constrained ML problems with large datasets are often limited to specific use cases, as stochastic algorithms for…
Given a convex set $\Omega$ of $\mathbb{R}^n$, we consider the shape optimization problem of finding a convex subset $\omega\subset \Omega$, of a given measure, minimizing the $p$-distance functional $$\mathcal{J}_p(\omega) :=…
We study the generalization properties of the popular stochastic optimization method known as stochastic gradient descent (SGD) for optimizing general non-convex loss functions. Our main contribution is providing upper bounds on the…
The Symmetric Information Bottleneck (SIB), an extension of the more familiar Information Bottleneck, is a dimensionality reduction technique that simultaneously compresses two random variables to preserve information between their…
Training-free guided sampling in diffusion models leverages off-the-shelf pre-trained networks, such as an aesthetic evaluation model, to guide the generation process. Current training-free guided sampling algorithms obtain the guidance…
We present a generic strategy iteration algorithm (GSIA) to find an optimal strategy of a simple stochastic game (SSG). We prove the correctness of GSIA, and derive a general complexity bound, which implies and improves on the results of…
We propose a new method for simplifying semidefinite programs (SDP) inspired by symmetry reduction. Specifically, we show if an orthogonal projection map satisfies certain invariance conditions, restricting to its range yields an equivalent…
Shell structures with a high stiffness-to-weight ratio are desirable in various engineering applications. In such scenarios, topology optimization serves as a popular and effective tool for shell structures design. Among the topology…
We present Flow-Induced Diagonal Gaussian Processes (FiD-GP), a compression framework that incorporates a compact inducing weight matrix to project a neural network's weight uncertainty into a lower-dimensional subspace. Critically, FiD-GP…