Related papers: Improving Solve Time of aggregation-based adaptive…
The Alternating Direction Method of Multipliers (ADMM) has now days gained tremendous attentions for solving large-scale machine learning and signal processing problems due to the relative simplicity. However, the two-block structure of the…
Mixture-of-Experts (MoE) models are designed to enhance the efficiency of large language models (LLMs) without proportionally increasing the computational demands. However, their deployment on edge devices still faces significant challenges…
In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentum, Adam) for a variety of vision problems (classification,…
In various scenarios, a single phase of modelling and solving is either not sufficient or not feasible to solve the problem at hand. A standard approach to solving AI planning problems, for example, is to incrementally extend the planning…
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with…
We propose an adaptive accelerated gradient method for solving smooth convex optimization problems. The method incorporates a scheme to determine the step size adaptively, by means of a local estimation of the smoothness constant, which is…
Science and engineering problems subject to uncertainty are frequently both computationally expensive and feature nonsmooth parameter dependence, making standard Monte Carlo too slow, and excluding efficient use of accelerated uncertainty…
This paper is concerned with the strong approximation of a semi-linear stochastic wave equation with strong damping, driven by additive noise. Based on a spatial discretization performed by a spectral Galerkin method, we introduce a kind of…
Hybrid CPU-GPU algorithms for Algebraic Multigrid methods (AMG) to efficiently utilize both CPU and GPU resources are presented. In particular, hybrid AMG framework focusing on minimal utilization of GPU memory with performance on par with…
We present a modification to the Berger and Oliger adaptive mesh refinement algorithm designed to solve systems of coupled, non-linear, hyperbolic and elliptic partial differential equations. Such systems typically arise during constrained…
Energy system models are challenged by the need for high temporal and spatial resolutions in or-der to appropriately depict the increasing share of intermittent renewable energy sources, storage technologies, and the growing…
Algebraic multigrid (AMG) methods derive their optimal efficiency from the interplay between a relaxation process and a corresponding coarse grid correction. In many standard formulations, relaxation and coarse-graining are analyzed and…
This paper studies the distributed model predictive control (DMPC) problem for distributed discrete-time linear systems with both local and global constraints over directed communication networks. We establish an optimization problem to…
Accurate uncertainty estimates can significantly improve the performance of iterative design of experiments, as in Sequential and Reinforcement learning. For many such problems in engineering and the physical sciences, the design task…
This chapter provides an overview of state-of-the-art adaptive finite element methods (AFEMs) for the numerical solution of second-order elliptic partial differential equations (PDEs), where the primary focus is on the optimal interplay of…
Improved EM strategies, based on the idea of efficient data augmentation (Meng and van Dyk 1997, 1998), are presented for ML estimation of mixture proportions. The resulting algorithms inherit the simplicity, ease of implementation, and…
We revisit the classical Local SGD and Federated Averaging (FedAvg) methods for distributed optimization and federated learning. While prior work has primarily focused on iteration complexity, we analyze these methods through the lens of…
Automatic segmentation of an image to identify all meaningful parts is one of the most challenging as well as useful tasks in a number of application areas. This is widely studied. Selective segmentation, less studied, aims to use limited…
This paper studies efficient distributed optimization methods for multi-agent networks. Specifically, we consider a convex optimization problem with a globally coupled linear equality constraint and local polyhedra constraints, and develop…
We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks. Our algorithm adds prior knowledge about the sequence of consecutive mini-batch gradients and…