Related papers: Generalized Landauer bound from absolute irreversi…

Since the celebrated works of Russo and Zou (2016,2019) and Xu and Raginsky (2017), it has been well known that the generalization error of supervised learning algorithms can be bounded in terms of the mutual information between their input…

Deep neural networks (DNNs) have significantly advanced machine learning, with model depth playing a central role in their successes. The dynamical system modeling approach has recently emerged as a powerful framework, offering new…

By using the coupling technique, we present sufficient conditions for the exponential ergodicity of general continuous-state nonlinear branching processes in both the $L^1$-Wasserstein distance and the total variation norm, where the drift…

We present a series of new and more favorable margin-based learning guarantees that depend on the empirical margin loss of a predictor. We give two types of learning bounds, both distribution-dependent and valid for general families, in…

We introduce a simple and straight-forward averaging procedure, which is a generalization of one which is commonly used in electrodynamics, and show that it possesses all the characteristics we require for linearized averaging in general…

Modern digital electronics support remarkably reliable computing, especially given the challenge of controlling nanoscale logical components that interact in fluctuating environments. However, we demonstrate that the high-reliability limit…

We present a new approach to far-from-equilibrium statistical mechanics, based on the concept of generalized entropy, which is a microscopically-defined generalization of Onsager-Machlup functional. In the case when a set of slow…

Inspired by recent strides in empirical efficacy of implicit learning in many robotics tasks, we seek to understand the theoretical benefits of implicit formulations in the face of nearly discontinuous functions, common characteristics for…

This paper introduces the generalized forward-backward splitting algorithm for minimizing convex functions of the form $F + \sum_{i=1}^n G_i$, where $F$ has a Lipschitz-continuous gradient and the $G_i$'s are simple in the sense that their…

We study the use of Gaussian process emulators to approximate the parameter-to-observation map or the negative log-likelihood in Bayesian inverse problems. We prove error bounds on the Hellinger distance between the true posterior…

A general achievable upper bound of extractable work under feedback control is given, where nonequilibrium equalities are generalized so as to be applicable to error-free measurements. The upper bound involves a term which arises from the…

We consider the correlations and the hydrodynamic description of random walkers with a general finite memory moving on a $d$ dimensional hypercubic lattice. We derive a drift-diffusion equation and identify a memory-dependent critical…

Examples with bound information on the regression function and density abound in many real applications. We propose a novel approach for estimating such functions by incorporating the prior knowledge on the bounds. Specially, a Gaussian…

The determination of the time averages of continuous functions, or discrete time sequences is important for various problems in physics and engineering, and the generalized final-value theorems of the Laplace and z-transforms, relevant to…

Understanding generalization is crucial to confidently engineer and deploy machine learning models, especially when deployment implies a shift in the data domain. For such domain adaptation problems, we seek generalization bounds which are…

Deep neural networks (DNNs) exhibit an exceptional capacity for generalization in practical applications. This work aims to capture the effect and benefits of depth for supervised learning via information-theoretic generalization bounds. We…

In this paper we define a class of coverage processes with infinitely divisible finite dimensional distributions and a particular type of correlation structure that can be thought of as generalizations of the classical Ornstein--Uhlenbeck…

We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They…

For a bipartite state with equal local dimension d, we prove that one can obtain work gain under Landauer's erasure process on one party in identically and independently distributed (iid) limit when the corresponding fully entangled…

Data-driven models are subject to model errors due to limited and noisy training data. Key to the application of such models in safety-critical domains is the quantification of their model error. Gaussian processes provide such a measure…