相关论文: Replacing K-infinity Function with Leaky ReLU in B…

We consider an independent and identically distributed (i.i.d.) random dynamical system of simple linear transformations on the unit interval $T_{\beta}(x)=\beta x$ (mod $1$), $x\in[0,1]$, $\beta>0$, which are the so-called…

We study layered neural networks of rectified linear units (ReLU) in a modelling framework for stochastic training processes. The comparison with sigmoidal activation functions is in the center of interest. We compute typical learning…

Efficient structural reanalysis for high-rank modification plays an important role in engineering computations which require repeated evaluations of structural responses, such as structural optimization and probabilistic analysis. To…

We study invariant measures for random countable (finite or infinite) conformal iterated function systems (IFS) with arbitrary overlaps. We do not assume any type of separation condition. We prove, under a mild assumption of finite entropy,…

Verifying the input-output relationships of a neural network so as to achieve some desired performance specification is a difficult, yet important, problem due to the growing ubiquity of neural nets in many engineering applications. We use…

We introduce a variational framework to learn the activation functions of deep neural networks. Our aim is to increase the capacity of the network while controlling an upper-bound of the actual Lipschitz constant of the input-output…

We consider the stability analysis of feedback systems with rectified linear unit (ReLU) activations, and model this problem with polynomial optimization. Stability can be certified by means of copositive multipliers in the framework of…

Activation functions are fundamental to deep neural networks, governing gradient flow, optimization stability, and representational capacity. Within historic deep architectures, while ReLU has been the dominant choice for the activation…

This paper is concerned with incremental stability properties of nonlinear systems. We propose conditions to compute an upper bound on the incremental L2-gain and to assess incremental asymptotic stability of piecewise-affine (PWA) systems.…

Despite the unresolved 'dying ReLU problem', the classical ReLU activation function (AF) has been extensively applied in Deep Neural Networks (DNN), in particular Convolutional Neural Networks (CNN), for image classification. The common…

We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions includes the much studied translation-invariant ones…

Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks. However, the benefits of novel activation functions have been inconsistent and task dependent, and…

We extend the ReLU Transition Graph (RTG) framework into a comprehensive graph-theoretic model for understanding deep ReLU networks. In this model, each node represents a linear activation region, and edges connect regions that differ by a…

Deep neural networks have recently achieved state-of-the-art results in many machine learning problems, e.g., speech recognition or object recognition. Hitherto, work on rectified linear units (ReLU) provides empirical and theoretical…

The goal of this paper is to test solids4Foam, the fluid-structure interaction (FSI) toolbox developed for foam-extend (a branch of OpenFOAM), and assess its flexibility in handling more complex flows. For this purpose, we consider the…

This work makes several contributions on stability and performance verification of nonlinear dynamical systems controlled by neural networks. First, we show that the stability and performance of a polynomial dynamical system controlled by a…

The Ensemble Kalman Filter method can be used as an iterative numerical scheme for parameter identification or nonlinear filtering problems. We study the limit of infinitely large ensemble size and derive the corresponding mean-field limit…

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes…

For problems involving large deformations of thin structures, simulating fluid-structure interaction (FSI) remains challenging largely due to the need to balance computational feasibility, efficiency, and solution accuracy. Overlapping…

A data-driven procedure for identifying the dominant transport barriers in a time-varying flow from limited quantities of Lagrangian data is presented. Our approach partitions state space into pairs of coherent sets, which are sets of…