Related papers: A feed-forward neural network as a nonlinear dynam…

Training deep neural networks (DNNs) in large-cluster computing environments is increasingly necessary, as networks grow in size and complexity. Local memory and processing limitations require robust data and model parallelism for crossing…

Model-based feedforward control improves tracking performance of motion systems, provided that the model describing the inverse dynamics is of sufficient accuracy. Model sets, such as neural networks (NNs) and physics-guided neural networks…

Molecular dynamics (MD) simulations are a central tool in science and engineering enabling the study of dynamical behavior and the link between microscopic structure and macroscopic function. Their high computational cost, however, has…

We study how neural networks trained by gradient descent extrapolate, i.e., what they learn outside the support of the training distribution. Previous works report mixed empirical results when extrapolating with neural networks: while…

Deep feedforward neural networks (DFNNs) are a powerful tool for functional approximation. We describe flexible versions of generalized linear and generalized linear mixed models incorporating basis functions formed by a DFNN. The…

In this paper, we systematically investigate both the synfire propagation and firing rate propagation in feedforward neuronal network coupled in an all-to-all fashion. In contrast to most earlier work, where only reliable synaptic…

Purpose: We propose a novel method for continual learning based on the increasing depth of neural networks. This work explores whether extending neural network depth may be beneficial in a life-long learning setting. Methods: We propose a…

This work presents a new algorithm for training recurrent neural networks (although ideas are applicable to feedforward networks as well). The algorithm is derived from a theory in nonconvex optimization related to the diffusion equation.…

The optimization problem behind neural networks is highly non-convex. Training with stochastic gradient descent and variants requires careful parameter tuning and provides no guarantee to achieve the global optimum. In contrast we show…

Feed-forward neural networks can be understood as a combination of an intermediate representation and a linear hypothesis. While most previous works aim to diversify the representations, we explore the complementary direction by performing…

We propose the feed-forward perturbation-based nonlinearity compensation method using the received signal, which outperforms conventional decision-based ones and eliminates the need for decision feedback. Additionally, combining half-half…

Due to their simple construction, LFSRs are commonly used as building blocks in various random number generators. Nonlinear feedforward logic is incorporated in LFSRs to increase the linear complexity of the generated sequence. In this…

As an imperative method of investigating the internal mechanism of femtosecond lasers, traditional femtosecond laser modeling relies on the split-step Fourier method (SSFM) to iteratively resolve the nonlinear Schrodinger equation suffering…

Graph neural networks (GNNs) have recently received significant attention. Learning node-wise message propagation in GNNs aims to set personalized propagation steps for different nodes in the graph. Despite the success, existing methods…

Modeling nonlinear pulse propagation in multimode fibers is challenging due to the large number of interacting modes and the resulting spatiotemporal complexity. Traditional optimization methods often become intractable, while…

Signal propagation in an optical fiber can be described by the nonlinear Schr\"odinger equation (NLSE). The NLSE has no known closed-form solution, mostly due to the interaction of dispersion and nonlinearities. In this paper, we present a…

A wide range of applications require learning image generation models whose latent space effectively captures the high-level factors of variation present in the data distribution. The extent to which a model represents such variations…

We introduce generative models for accelerating simulations of complex systems through learning and evolving their effective dynamics. In the proposed Generative Learning of Effective Dynamics (G-LED), instances of high dimensional data are…

We utilize the Feature Decoupling Distributed (FDD) method to enhance the capability of deep learning to fit the Nonlinear Schrodinger Equation (NLSE), significantly reducing the NLSE loss compared to non decoupling model.

Simulating the long-timescale dynamics of biomolecules is a central challenge in computational science. While enhanced sampling methods can accelerate these simulations, they rely on pre-defined collective variables that are often difficult…