Related papers: The Global Structure of Codimension-2 Local Bifurc…

Recurrent neural networks (RNNs) are widely used throughout neuroscience as models of local neural activity. Many properties of single RNNs are well characterized theoretically, but experimental neuroscience has moved in the direction of…

This work develops the global equations of neural networks through stacked piecewise manifolds, fixed-point theory, and boundary-conditioned iteration. Once fixed coordinates and operators are removed, a neural network appears as a…

Task-based modeling with recurrent neural networks (RNNs) has emerged as a popular way to infer the computational function of different brain regions. These models are quantitatively assessed by comparing the low-dimensional neural…

The saddle-node bifurcation is the simplest example of a generic bifurcation in smooth ordinary differential equations, and is associated with the creation or destruction of a pair of equilibria. In this paper we examine the unfolding of…

Recurrent neural networks (RNNs) have proved effective at one dimensional sequence learning tasks, such as speech and online handwriting recognition. Some of the properties that make RNNs suitable for such tasks, for example robustness to…

Circuits of biological neurons, such as in the functional parts of the brain can be modeled as networks of coupled oscillators. Inspired by the ability of these systems to express a rich set of outputs while keeping (gradients of) state…

Protein secondary structure prediction is an important problem in bioinformatics. Inspired by the recent successes of deep neural networks, in this paper, we propose an end-to-end deep network that predicts protein secondary structures from…

For piecewise-linear maps, the phenomenon that a branch of a one-dimensional unstable manifold of a periodic solution is completely contained in its stable manifold is codimension-two. Unlike codimension-one homoclinic corners, such…

Existing deep convolutional neural networks (CNNs) have shown their great success on image classification. CNNs mainly consist of convolutional and pooling layers, both of which are performed on local image areas without considering the…

Neural population activity in sensory cortex is organized on low-dimensional manifolds, but why such manifolds arise and what determines their geometry remain unclear. We model cortical populations as recurrent circuits driven by…

The integration and transmission of information in the brain are dependent on the interplay between structural and dynamical properties. Implicit in any pursuit aimed at understanding neural dynamics from appropriate sets of mathematically…

We establish center manifold theorems that allow one to study the bifurcation of small solutions from a trivial state in systems of functional equations posed on the real line. The class of equations includes most importantly nonlinear…

We unfold the codimension-two simultaneous occurrence of a border-collision bifurcation and a period-doubling bifurcation for a general piecewise-smooth, continuous map. We find that, with sufficient non-degeneracy conditions, a locus of…

Learning algorithms for natural language processing (NLP) tasks traditionally rely on manually defined relevant contextual features. On the other hand, neural network models using an only distributional representation of words have been…

Periodic normal forms for the codim 2 bifurcations of limit cycles up to a 3-dimensional center manifold in generic autonomous ODEs and computational formulas for their coefficients are derived. The formulas are independent of the dimension…

Despite significant advances in the field of deep learning in ap-plications to various areas, an explanation of the learning pro-cess of neural network models remains an important open ques-tion. The purpose of this paper is a comprehensive…

Deep Recurrent Neural Network architectures, though remarkably capable at modeling sequences, lack an intuitive high-level spatio-temporal structure. That is while many problems in computer vision inherently have an underlying high-level…

We discuss the bifurcation structure of homoclinic orbits in bimodal one dimensional maps. The universal structure of these bifurcations with singular bifurcation points and the web of bifurcation lines through the parameter space are…

Models based on deep convolutional networks have dominated recent image interpretation tasks; we investigate whether models which are also recurrent, or "temporally deep", are effective for tasks involving sequences, visual and otherwise.…

A systematic study of closed classical orbits of the hydrogen atom in crossed electric and magnetic fields is presented. We develop a local bifurcation theory for closed orbits which is analogous to the well-known bifurcation theory for…