Related papers: Analyzing Echo-state Networks Using Fractal Dimens…
As one of the most important paradigms of recurrent neural networks, the echo state network (ESN) has been applied to a wide range of fields, from robotics to medicine, finance, and language processing. A key feature of the ESN paradigm is…
Learning to solve sequential tasks with recurrent models requires the ability to memorize long sequences and to extract task-relevant features from them. In this paper, we study the memorization subtask from the point of view of the design…
Fractal dimension is defined on the base of entropy, including macro state entropy and information entropy. The generalized correlation dimension of multifractals is based on Renyi entropy. However, the mathematical transform from entropy…
Fractal dimension constitutes the main tool to test for fractal patterns in Euclidean contexts. For this purpose, it is always used the box dimension, since it is easy to calculate, though the Hausdorff dimension, which is the oldest and…
To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular,…
A key attribute that drives the unprecedented success of modern Recurrent Neural Networks (RNNs) on learning tasks which involve sequential data, is their ability to model intricate long-term temporal dependencies. However, a well…
We show that fractality in complex networks arises from the geometric self-similarity of their built-in hierarchical community-like structure, which is mathematically described by the scale-invariant equation for the masses of the boxes…
In this paper we present a computational model which decodes the spatio-temporal data from electro-physiological measurements of neuronal networks and reconstructs the network structure on a macroscopic domain, representing the connectivity…
Reservoir Computing (RC) has become popular in recent years thanks to its fast and efficient computational capabilities. Standard RC has been shown to be equivalent in the asymptotic limit to Recurrent Kernels, which helps in analyzing its…
Training recurrent neural networks (RNNs) is a hard problem due to degeneracies in the optimization landscape, a problem also known as vanishing/exploding gradients. Short of designing new RNN architectures, previous methods for dealing…
We introduce the concept of boundaries of a complex network as the set of nodes at distance larger than the mean distance from a given node in the network. We study the statistical properties of the boundaries nodes of complex networks. We…
The effectiveness of recurrent neural networks can be largely influenced by their ability to store into their dynamical memory information extracted from input sequences at different frequencies and timescales. Such a feature can be…
The improved city clustering algorithm can be used to identify urban boundaries on a digital map, and the results are a set of isolines. The relationships between the urban measurements within the variable boundaries follow allometric…
In the realm of fractal geometry, intricate structures emerge from simple iterative processes that partition parameter spaces into regions of stability and instability. Likewise, training large language models involves iteratively applying…
Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which can be trained. Reversible RNNs---RNNs for which the…
As an efficient recurrent neural network (RNN) model, reservoir computing (RC) models, such as Echo State Networks, have attracted widespread attention in the last decade. However, while they have had great success with time series data…
Recently, frequency security is challenged by high uncertainty and low inertia in power system with high penetration of Renewable Energy Sources (RES). In the context of Unit Commitment (UC) problems, frequency security constraints…
In a previous paper, we have shown that a recurrent neural network (RNN) can be used to detect cellular network radio signal degradations accurately. We unexpectedly found, though, that accuracy gains diminished as we added layers to the…
The spectral dimension has been widely used to understand transport properties on regular and fractal lattices. Nevertheless, it has been little studied for complex networks such as scale-free and small world networks. Here we study the…
The effect of geometry and morphology of superconducting structure on magnetic flux trapping is considered. It is found that the clusters of normal phase, which act as pinning centers, have significant fractal properties. The fractal…