Related papers: Analyzing Echo-state Networks Using Fractal Dimens…

Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance. But does better performance necessarily imply better internal representations? While the representational…

Reservoir computing, using nonlinear dynamical systems, offers a cost-effective alternative to neural networks for complex tasks involving processing of sequential data, time series modeling, and system identification. Echo state networks…

We consider the problem of neural association for a network of non-binary neurons. Here, the task is to first memorize a set of patterns using a network of neurons whose states assume values from a finite number of integer levels. Later,…

Deep Reservoir Computing has emerged as a new paradigm for deep learning, which is based around the reservoir computing principle of maintaining random pools of neurons combined with hierarchical deep learning. The reservoir paradigm…

We consider an \textit{Adaptive Random Convolutional Network Coding} (ARCNC) algorithm to address the issue of field size in random network coding for multicast, and study its memory and decoding delay performances through both analysis and…

In this work, we study the fractal and multifractal properties of a family of fractal networks introduced by Gallos {\it et al.} ({\it Proc. Natl. Acad. Sci. U.S.A.}, 2007, {\bf 104}: 7746). In this fractal network model, there is a…

We present a framework for analyzing what the state in RNNs remembers from its input embeddings. Our approach is inspired by backpropagation, in the sense that we compute the gradients of the states with respect to the input embeddings. The…

A fractal is in essence a hierarchy with cascade structure, which can be described with a set of exponential functions. From these exponential functions, a set of power laws indicative of scaling can be derived. Hierarchy structure and…

The fractal dimension of domain walls produced by changing the boundary conditions from periodic to anti-periodic in one spatial direction is studied using both the strong-disorder renormalization group and the greedy algorithm for the…

Reservoir computers (RC) have proven useful as surrogate models in forecasting and replicating systems of chaotic dynamics. The quality of surrogate models based on RCs is crucially dependent on their optimal implementation that involves…

Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a "next-generation"…

Recurrent neural networks (RNNs) are a vital modeling technique that rely on internal states learned indirectly by optimization of a supervised, unsupervised, or reinforcement training loss. RNNs are used to model dynamic processes that are…

A class of simplified measures is constructed to capture the key features of generic spatio-temporally chaotic systems. A combined analytical and numerical investigation allows us to extablish the scaling beahviour of the fractal dimension…

Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics. This sets them apart from deep feed-forward networks. Despite the tremendous progress in the application of feed-forward networks and their theoretical…

Recently, deep convolutional neural networks (CNNs) have achieved many eye-catching results. However, deploying CNNs on resource-constrained edge devices is constrained by limited memory bandwidth for transmitting large intermediated data…

Echo State Networks represent a type of recurrent neural network with a large randomly generated reservoir and a small number of readout connections trained via linear regression. The most common topology of the reservoir is a fully…

We explore the problem of finding the Hausdorff dimension of the set of points that recur to shrinking targets on a self-affine fractal. To be exact, we study the dimension of a certain related symbolic recurrence set. In many cases this…

Reservoir computing (RC) offers efficient temporal data processing with a low training cost by separating recurrent neural networks into a fixed network with recurrent connections and a trainable linear network. The quality of the fixed…

Semantic segmentation has recently witnessed major progress, where fully convolutional neural networks have shown to perform well. However, most of the previous work focused on improving single image segmentation. To our knowledge, no prior…

Deep neural networks (DNNs) are so over-parametrized that recent research has found them to already contain a subnetwork with high accuracy at their randomly initialized state. Finding these subnetworks is a viable alternative training…