Related papers: An introduction to memory competitions, records an…
This article deals with dynamical systems depending on a slowly varying parameter. We present several physical examples illustrating memory effects, such as metastability and hysteresis, which frequently appear in these systems. A…
Existing reasoning tasks often have an important assumption that the input contents can be always accessed while reasoning, requiring unlimited storage resources and suffering from severe time delay on long sequences. To achieve efficient…
We study numerically the memory which forgets, introduced in 1986 by Parisi by bounding the synaptic strength, with a mechanism which avoid confusion, allows to remember the pattern learned more recently and has a physiologically very well…
Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions. This paper studies the impact of multitask and transfer learning for simple question…
In the process of collectively inventing new words for new concepts in a population, conflicts can quickly become numerous, in the form of synonymy and homonymy. Remembering all of them could cost too much memory, and remembering too few…
What happen in the brain when human beings play games with computers? Here a simple zero-sum game was conducted to investigate how people make decision via their brain even they know that their opponent is a computer. There are two choices…
To be effective in sequential data processing, Recurrent Neural Networks (RNNs) are required to keep track of past events by creating memories. While the relation between memories and the network's hidden state dynamics was established over…
Recently, flash memories have become a competitive solution for mass storage. The flash memories have rather different properties compared with the rotary hard drives. That is, the writing of flash memories is constrained, and flash…
Scientific and technological progress is largely driven by firms in many domains, including artificial intelligence and vaccine development. However, we do not know yet whether the success of firms' research activities exhibits dynamic…
Popularity describes the dynamics of mass attention, and is a part of a broader class of population dynamics in ecology and social science literature. Studying accurate model of popularity is important for quantifying spreading of novelty,…
The training of modern large language models (LLMs) takes place in a regime where most training examples are seen only a few times by the model during the course of training. What does a model remember about such examples seen only a few…
Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image classification, voice recognition, translation, and other tasks. But this progress has come with…
Despite their wide adoption, the underlying training and memorization dynamics of very large language models is not well understood. We empirically study exact memorization in causal and masked language modeling, across model sizes and…
This study investigates the effect of competition between individuals on population dynamics when they compete for different resources during different seasons or during different growth stages. Individuals are assumed to compete for a…
For many years, researchers in psychology, education, statistics, and machine learning have been developing practical methods to improve learning speed, retention, and generalizability, and this work has been successful. Many of these…
Nowadays, Computer Science tightly entered all spheres of human activity. To improve quality and speed of development process, it is important to help programmers improve their working conditions. This paper proposes a vision on exploring…
Inspired by the Games held in ancient Greece, modern Olympics represent the world's largest pageant of athletic skill and competitive spirit. Performances of athletes at the Olympic Games mirror, since 1896, human potentialities in sports,…
Recent advancements in data-to-text generation largely take on the form of neural end-to-end systems. Efforts have been dedicated to improving text generation systems by changing the order of training samples in a process known as…
We start with the well-known game below: Two players hold a sheet of paper to their forehead on which a positive integer is written. The numbers are consecutive and each player can only see the number of the other one. In each time step,…
Memory, a fundamental component of human cognition, exhibits adaptive yet fallible characteristics as illustrated by Schacter's memory "sins".These cognitive phenomena have been studied extensively in psychology and neuroscience, but the…