Related papers: Generalization Bounds for Stochastic Saddle Point …

In this paper, we consider the problem of finding an almost surely common fixed point of a family of paracontraction maps indexed on a probability space, which we refer to as the stochastic feasibility problem. We show that a random…

Stochastic Gradient Descent (SGD) is a widely deployed optimization procedure throughout data-driven and simulation-driven disciplines, which has drawn a substantial interest in understanding its global behavior across a broad class of…

We study smoothness of generalized solutions of nonlocal elliptic problems in plane bounded domains with piecewise smooth boundary. The case where the support of nonlocal terms can intersect the boundary is considered. We announce…

Algorithmic stability is among the most potent techniques in generalization analysis. However, its derivation usually requires a stepsize $\eta_t = \mathcal{O}(1/t)$ under non-convex training regimes, where $t$ denotes iterations. This…

In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a bound on the…

We investigate the feasibility of sample average approximation (SAA) for general stochastic optimization problems, including two-stage stochastic programming without the relatively complete recourse assumption. Instead of analyzing problems…

We study linear stochastic partial differential equations of parabolic type. We consider a new boundary value problem where a Cauchy condition is replaced by a prescribed average of the solution either over time and probabilistic space for…

We study a stochastic first order primal-dual method for solving convex-concave saddle point problems over real reflexive Banach spaces using Bregman divergences and relative smoothness assumptions, in which we allow for stochastic error in…

This paper investigates stochastic generalized dynamic games with coupling chance constraints, where agents have incomplete information about uncertainties satisfying a concentration of measure property. This problem, in general, is…

A State Space Model (SSM) is a foundation model in time series analysis, which has recently been shown as an alternative to transformers in sequence modeling. In this paper, we theoretically study the generalization of SSMs and propose…

Building upon the results in [Hinterm\"uller et al., SIAM J. Optim, '15], generalized Nash equilibrium problems are considered, in which the feasible set of each player is influenced by the decisions of their competitors. This is realized…

Uniform stability of a learning algorithm is a classical notion of algorithmic stability introduced to derive high-probability bounds on the generalization error (Bousquet and Elisseeff, 2002). Specifically, for a loss function with range…

In this paper, we investigate a class of constrained saddle point (SP) problems where the objective function is nonconvex-concave and smooth. This class of problems has wide applicability in machine learning, including robust multi-class…

This paper presents a comprehensive analysis of a broad range of variations of the stochastic proximal point method (SPPM). Proximal point methods have attracted considerable interest owing to their numerical stability and robustness…

In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we assume that each node has access to a summand of a global…

In this paper we prove a new abstract stability result for perturbed saddle-point problems based on a norm fitting technique. We derive the stability condition according to Babuska's theory from a small inf-sup condition, similar to the…

We develop a mathematically and physically sound definition of the spectrally-hyperviscous Navier-Stokes equations (SHNSE) on general bounded domains \Omega with zero (no-slip) boundary conditions prescribed on \varGamma=\partial\varOmega.…

Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency. For a fixed stepsize, the iterates typically admit a stationary distribution that is rarely tractable. Prior work shows that as the…

We study the generalization error of randomized learning algorithms -- focusing on stochastic gradient descent (SGD) -- using a novel combination of PAC-Bayes and algorithmic stability. Importantly, our generalization bounds hold for all…

Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimization properties of SGDM have been extensively studied in the literature, it remains insufficiently…