Related papers: Stochastic Chaining and Strengthened Information-T…

We present a new method for stochastic shape optimisation of engineering structures. The method generalises an existing deterministic scheme, in which the structure is represented and evolved by a level-set method coupled with mathematical…

Coarse-graining techniques play a central role in reducing the complexity of stochastic models, and are typically characterised by a mapping which projects the full state of the system onto a smaller set of variables which captures the…

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…

Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the dataset and the training algorithm must be taken into account…

A widely used heuristic for solving stochastic optimization problems is to use a deterministic rolling horizon procedure, which has been modified to handle uncertainty (e.g. buffer stocks, schedule slack). This approach has been criticized…

When looking for a solution, deterministic methods have the enormous advantage that they do find global optima. Unfortunately, they are very CPU-intensive, and are useless on untractable NP-hard problems that would require thousands of…

Training modern neural networks often relies on large learning rates, operating at the edge of stability, where the optimization dynamics exhibit oscillatory and chaotic behavior. Empirically, this regime often yields improved…

One of the fundamental challenges in the deep learning community is to theoretically understand how well a deep neural network generalizes to unseen data. However, current approaches often yield generalization bounds that are either too…

Sequential quadratic optimization algorithms are proposed for solving smooth nonlinear optimization problems with equality constraints. The main focus is an algorithm proposed for the case when the constraint functions are deterministic,…

The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…

Providing generalization guarantees for stochastic optimization algorithms remains a key challenge in learning theory. Recently, numerous works demonstrated the impact of the geometric properties of optimization trajectories on…

A framework previously introduced in [3] for solving a sequence of stochastic optimization problems with bounded changes in the minimizers is extended and applied to machine learning problems such as regression and classification. The…

Stochastic optimisation algorithms are the de facto standard for machine learning with large amounts of data. Handling only a subset of available data in each optimisation step dramatically reduces the per-iteration computational costs,…

In this paper, a robust sequential quadratic programming method for constrained optimization is generalized to problem with an {expectation} objective function {and} deterministic equality and inequality constraints. A stochastic line…

Variant Stochastic cracking is a significantly more resilient approach to adaptive indexing. It showed [1]that Stochastic cracking uses each query as a hint on how to reorganize data, but not blindly so; it gains resilience and avoids…

Coarse-grained models of chaotic systems neglect unresolved degrees of freedom, inducing structured model error that limits predictability and distorts long-term statistics. Typical data-driven closures are trained to minimize error over a…

We investigate the in-distribution generalization of machine learning algorithms. We depart from traditional complexity-based approaches by analyzing information-theoretic bounds that quantify the dependence between a learning algorithm and…

In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a bound on the…

This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infinity, its predictive variance on the training set decreases…

During the last few years, significant attention has been paid to the stochastic training of artificial neural networks, which is known as an effective regularization approach that helps improve the generalization capability of trained…