Related papers: Stability of implicit neural networks for long-ter…
The E(xplicit)I(implicit)N(null) method was developed recently to remove numerical instability from PDEs, adding and subtracting an operator $\mathcal{D}$ of arbitrary structure, treating the operator implicitly in one case, and explicitly…
This work investigates the exponential stability of neural networks (NNs) systems with time delays. By considering orthogonal polynomials with weighted terms, a new weighted integral inequality is presented. This inequality extend several…
We consider the application of deep generative models in propagating uncertainty through complex physical systems. Specifically, we put forth an implicit variational inference formulation that constrains the generative model output to…
Robust physics (e.g., governing equations and laws) discovery is of great interest for many engineering fields and explainable machine learning. A critical challenge compared with general training is that the term and format of governing…
Neural networks have become increasingly popular in controller design due to their versatility and efficiency. However, their integration into feedback systems can pose stability challenges, particularly in the presence of uncertainties.…
Inference for mechanistic models is challenging because of nonlinear interactions between model parameters and a lack of identifiability. Here we focus on a specific class of mechanistic models, which we term stable differential equations.…
Arguably the biggest challenge in applying neural networks is tuning the hyperparameters, in particular the learning rate. The sensitivity to the learning rate is due to the reliance on backpropagation to train the network. In this paper we…
The stability of classical semi-implicit scheme, and some more advanced iterative schemes recently proposed for Numerical Weather Prediction (NWP) purpose is examined. In all these schemes, the solution of the centred-implicit non-linear…
We examine the stability of loss-minimizing training processes that are used for deep neural networks (DNN) and other classifiers. While a classifier is optimized during training through a so-called loss function, the performance of…
Partial Differential Equations (PDEs) with high dimensionality are commonly encountered in computational physics and engineering. However, finding solutions for these PDEs can be computationally expensive, making model-order reduction…
Implicit equilibrium models, i.e., deep neural networks (DNNs) defined by implicit equations, have been becoming more and more attractive recently. In this paper, we investigate an emerging question: can an implicit equilibrium model's…
In many important machine learning applications, the training distribution used to learn a probabilistic classifier differs from the testing distribution on which the classifier will be used to make predictions. Traditional methods correct…
Sparse deep learning has become a popular technique for improving the performance of deep neural networks in areas such as uncertainty quantification, variable selection, and large-scale network compression. However, most existing research…
Deep learning systems are known to exhibit implicit regularization (alt. implicit bias), favoring simple solutions instead of merely minimizing the loss function. In some cases, we can analytically derive the implicit regularization --…
We introduce a deep residual recurrent neural network (DR-RNN) as an efficient model reduction technique for nonlinear dynamical systems. The developed DR-RNN is inspired by the iterative steps of line search methods in finding the residual…
Accurate and efficient simulations of physical phenomena governed by partial differential equations (PDEs) are important for scientific and engineering progress. While traditional numerical solvers are powerful, they are often…
Traditional approaches to stabilizing hyperbolic PDEs, such as PDE backstepping, often encounter challenges when dealing with high-dimensional or complex nonlinear problems. Their solutions require high computational and analytical costs.…
Multiscale stochastic dynamical systems have been widely adopted to a variety of scientific and engineering problems due to their capability of depicting complex phenomena in many real world applications. This work is devoted to…
Deep Neural Networks (DNNs) are becoming integral components of real world services relied upon by millions of users. Unfortunately, architects of these systems can find it difficult to ensure reliable performance as irrelevant details like…
Neural ordinary differential equations (Neural ODEs) propose the idea that a sequence of layers in a neural network is just a discretisation of an ODE, and thus can instead be directly modelled by a parameterised ODE. This idea has had…