Related papers: R\'{e}seaux d'Automates de Caianiello Revisit\'{e}

We characterize the structure of the periods of a neuronal recurrence equation. Firstly, we give a characterization of k-chains in 0-1 periodic sequences. Secondly, we characterize the periods of all cycles of some neuronal recurrence…

Do neural networks, trained on well-understood algorithmic tasks, reliably rediscover known algorithms for solving those tasks? Several recent studies, on tasks ranging from group arithmetic to in-context linear regression, have suggested…

Let $0<k\in\mathbb{Z}$. A reinterpretation of the proof of existence of Hamilton cycles in the middle-levels graph $M_k$ induced by the vertices of the $(2k+1)$-cube representing the $k$- and $(k+1)$-subsets of $\{0,\ldots,2k\}$ is given…

We present a detailed description of the diagrammatic technique, recently devised in [V. V. Kuzmin et.al., npj Quantum Information 5, 115 (2019)], for semi-analytical description of large-scale quantum-repeater networks. The technique takes…

We provide a comprehensive review of the existing literature on memory-augmented GNNs. We review these works through the lens of psychology and neuroscience, which has several established theories on how multiple memory systems and…

Free-running Recurrent Neural Networks (RNNs), especially probabilistic models, generate an ongoing information flux that can be quantified with the mutual information $I\left[\vec{x}(t),\vec{x}(t\!+\!1)\right]$ between subsequent system…

It is shown that a hamiltonian $n/2$-regular bipartite graph $G$ of order $2n>8$ contains a cycle of length $2n-2$. Moreover, if such a cycle can be chosen to omit a pair of adjacent vertices, then $G$ is bipancyclic.

Recurrent neural networks (RNNs) are a class of neural networks used in sequential tasks. However, in general, RNNs have a large number of parameters and involve enormous computational costs by repeating the recurrent structures in many…

One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…

Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which can be trained. Reversible RNNs---RNNs for which the…

Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is…

We train a small message-passing graph neural network to predict Hamiltonian cycles on Erd\H{o}s-R\'enyi random graphs in a critical regime. It outperforms existing hand-crafted heuristics after about 2.5 hours of training on a single GPU.…

We review neural network architectures which were motivated by Fourier series and integrals and which are referred to as Fourier neural networks. These networks are empirically evaluated in synthetic and real-world tasks. Neither of them…

This is a tutorial paper on Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), and their variants. We start with a dynamical system and backpropagation through time for RNN. Then, we discuss the problems of gradient…

This work attempts to explain the types of computation that neural networks can perform by relating them to automata. We first define what it means for a real-time network with bounded precision to accept a language. A measure of network…

We study the ability of linear recurrent networks obeying discrete time dynamics to store long temporal sequences that are retrievable from the instantaneous state of the network. We calculate this temporal memory capacity for both…

The Fibonacci sequence is a sequence of numbers that has been studied for hundreds of years. In this paper, we introduce the new sequence S_{k,n} with initial conditions S_{k,0} = 2b and S_{k,1} = bk + a, which is generated by the…

In this paper, we consider the problem of finding a cycle of length $2k$ (a $C_{2k}$) in an undirected graph $G$ with $n$ nodes and $m$ edges for constant $k\ge2$. A classic result by Bondy and Simonovits [J.Comb.Th.'74] implies that if $m…

Neuromorphic hardware platforms, such as Intel's Loihi chip, support the implementation of Spiking Neural Networks (SNNs) as an energy-efficient alternative to Artificial Neural Networks (ANNs). SNNs are networks of neurons with internal…

Let ${\mathcal C}$ be a proper minor-closed family of graphs. We present a randomized algorithm that given a graph $G \in {\mathcal C}$ with $n$ vertices, finds a simple cycle of size $k$ in $G$ (if exists) in $2^{O(k)}n$ time. The…