Related papers: LDP for the covariance process in fully connected …
We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant $n$. Under mild assumptions on the non-linearity, we obtain…
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers. DGPs are probabilistic and non-parametric…
We prove a large deviation principle for a greedy exploration process on an Erd\"os-R\'enyi (ER) graph when the number of nodes goes to infinity. To prove our main result, we use the general strategy to study large deviations of processes…
We propose a network structure discovery model for continuous observations that generalizes linear causal models by incorporating a Gaussian process (GP) prior on a network-independent component, and random sparsity and weight matrices as…
We investigate possible large deviation principles (LDPs) for the $n$-vertex sampling from a given graphon with various speeds $s(n)$ and resolve all the cases except when the speed $s(n)$ is of order $n^2$. For quadratic speed…
The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing…
We study an inhomogeneous sparse random graph on [N] = {1, . . . , N } as introduced in a seminal paper by Bollobas, Janson and Riordan (2007): vertices have a type (here in a compact metric space S), and edges between different vertices…
We consider a system of stochastic interacting particles in $\mathbb{R}^d$ and we describe large deviations asymptotics in a joint mean-field and small-noise limit. Precisely, a large deviations principle (LDP) is established for the…
Graph convolutional neural networks~(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification, but little work has been done to explore their theoretical properties. Recently, several deep neural…
This paper is the second in the series Commutative Scaling of Width and Depth (WD) about commutativity of infinite width and depth limits in deep neural networks. Our aim is to understand the behaviour of neural functions (functions that…
Dirichlet processes (DP) are widely applied in Bayesian nonparametric modeling. However, in their basic form they do not directly integrate dependency information among data arising from space and time. In this paper, we propose location…
There is a previously identified equivalence between wide fully connected neural networks (FCNs) and Gaussian processes (GPs). This equivalence enables, for instance, test set predictions that would have resulted from a fully Bayesian,…
Study of neural networks with infinite width is important for better understanding of the neural network in practical application. In this work, we derive the equivalence of the deep, infinite-width maxout network and the Gaussian process…
This study focuses on large deviation principles for fully coupled multiscale multivalued stochastic systems, in which the slow component is governed by a multivalued stochastic differential equation and the fast component is described by a…
The theory of large deviations has been applied successfully in the last 30 years or so to study the properties of equilibrium systems and to put the foundations of equilibrium statistical mechanics on a clearer and more rigorous footing. A…
Link prediction aims to reveal missing edges in a graph. We address this task with a Gaussian process that is transformed using simplified graph convolutions to better leverage the inductive bias of the domain. To scale the Gaussian process…
Stochastic gradient descent (SGD) and its variants have established themselves as the go-to algorithms for large-scale machine learning problems with independent samples due to their generalization performance and intrinsic computational…
We establish, under the Cramer exponential moment condition in a neighbourhood of zero, the Extended Large Deviation Principle for the Random Walk and the Compound Poisson processes in the metric space $\V$ of functions of finite variation…
This paper is concerned with a state-space approach to deep Gaussian process (DGP) regression. We construct the DGP by hierarchically putting transformed Gaussian process (GP) priors on the length scales and magnitudes of the next level of…
Deep Gaussian processes (DGP) have appealing Bayesian properties, can handle variable-sized data, and learn deep features. Their limitation is that they do not scale well with the size of the data. Existing approaches address this using a…