Related papers: The First Principle of Big Memory Systems
Catastrophic forgetting affects the training of neural networks, limiting their ability to learn multiple tasks sequentially. From the perspective of the well established plasticity-stability dilemma, neural networks tend to be overly…
Continuous first-order logic is used to apply model-theoretic analysis to analytic structures (e.g. Hilbert spaces, Banach spaces, probability spaces, etc.). Classical computable model theory is used to examine the algorithmic structure of…
Despite advances in deep learning, neural networks can only learn multiple tasks when trained on them jointly. When tasks arrive sequentially, they lose performance on previously learnt tasks. This phenomenon called catastrophic forgetting…
Persistence, defined as the probability that a fluctuating signal has not reached a threshold up to a given observation time, plays a crucial role in the theory of random processes. It quantifies the kinetics of processes as varied as phase…
Persistence modules are representations of products of totally ordered sets in the category of vector spaces. They appear naturally in the representation theory of algebras, but in recent years they have also found applications in other…
The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions…
In order to solve complex configuration tasks in technical domains, various knowledge based methods have been developed. However their applicability is often unsuccessful due to their low efficiency. One of the reasons for this is that…
Informatics and technological advancements have triggered generation of huge volume of data with varied complexity in its management and analysis. Big Data analytics is the practice of revealing hidden aspects of such data and making…
Stochastic neural networks are a prototypical computational device able to build a probabilistic representation of an ensemble of external stimuli. Building on the relationship between inference and learning, we derive a synaptic plasticity…
We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in…
Many social and biological systems are characterized by enduring hierarchies, including those organized around prestige in academia, dominance in animal groups, and desirability in online dating. Despite their ubiquity, the general…
Persistence and permanence are properties of dynamical systems that describe the long-term behavior of the solutions, and in particular specify whether positive solutions approach the boundary of the positive orthant. Mass-action systems…
Resistive memories are outstanding electron devices that have displayed a large potential in a plethora of applications such as nonvolatile data storage, neuromorphic computing, hardware cryptography, etc. Their fabrication control and…
Artificial intelligence (AI) research today is largely driven by ever-larger neural network models trained on graphics processing units (GPUs). This paradigm has yielded remarkable progress, but it also risks entrenching a hardware lottery…
Computer systems can be found everywhere: in space, in our homes, in our cars, in our pockets, and sometimes even in our own bodies. For concerns of safety, economy, and convenience, it is important that such systems work correctly.…
This new long version of my 1983 paper suggests the goals you might have for your system -- Simple, Timely, Efficient, Adaptable, Dependable, Yummy (STEADY) -- and techniques for achieving them -- Approximate, Incremental, Divide & Conquer…
Several systems can be represented by hypergraphs, an extension of graphs with associations between any number of vertices. These natural hypergraphs doe not appear at once. They are generated by some dynamical process of hypergraph…
The demand for high-density data storage with ultrafast accessibility motivates the search for new memory implementations. Ideally such storage devices should be robust to input error and to unreliability of individual elements; furthermore…
The efficient simulation of complex quantum systems remains a central challenge due to the exponential growth of Hilbert space with system size. Tensor network methods have long been established as powerful approximation schemes, and their…
Advances in materials science have led to physical instantiations of self-assembled networks of memristive devices and demonstrations of their computational capability through reservoir computing. Reservoir computing is an approach that…