Related papers: Cache-Oblivious Persistence
Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian…
We propose a general data structure CORoBTS for storing B-tree-like search trees dynamically in a cache-oblivious way combining the van Emde Boas memory layout with packed memory array. In the use of the vEB layout mostly search complexity…
Large language models have revolutionized data processing in numerous domains, with their ability to handle extended context reasoning receiving notable recognition. To speed up inference, maintaining a key-value (KV) cache memory is…
We use the law of total variance to generate multiple expressions for the posterior predictive variance in Bayesian hierarchical models. These expressions are sums of terms involving conditional expectations and conditional variances. Since…
Learning new information without forgetting prior knowledge is central to human intelligence. In contrast, neural network models suffer from catastrophic forgetting: a significant degradation in performance on previously learned tasks when…
Learning depends on the ability to acquire and assimilate new information. This ability depends---somewhat counterintuitively---on the ability to forget. In particular, effective forgetting requires the ability to recognize and utilize new…
In a recent paper Chan et al. [SODA '19] proposed a relaxation of the notion of (full) memory obliviousness, which was introduced by Goldreich and Ostrovsky [J. ACM '96] and extensively researched by cryptographers. The new notion,…
We present data-oblivious algorithms in the external-memory model for compaction, selection, and sorting. Motivation for such problems comes from clients who use outsourced data storage services and wish to mask their data access patterns.…
We give an example of a sequential dynamical system consisting of intermittent-type maps which exhibits loss of memory with a polynomial rate of decay. A uniform bound holds for the upper rate of memory loss. The maps may be chosen in any…
Data is one of the most important factors in machine learning. However, even if we have high-quality data, there is a situation in which access to the data is restricted. For example, access to the medical data from outside is strictly…
We consider a basic model of digital memory where each cell is composed of a reflecting medium with two possible reflectivities. By fixing the mean number of photons irradiated over each memory cell, we show that a non-classical source of…
Huge memory consumption has been a major bottleneck for deploying high-throughput large language models in real-world applications. In addition to the large number of parameters, the key-value (KV) cache for the attention mechanism in the…
Attention mechanisms are a central property of cognitive systems allowing them to selectively deploy cognitive resources in a flexible manner. Attention has been long studied in the neurosciences and there are numerous phenomenological…
Forgetfulness is a common feature of nature. Moreover, without forgetfulness, repeatability would be impossible. Despite this, small systems constantly leak information about their state to their surroundings, and quantum mechanics tells us…
Recently, continuous cache models were proposed as extensions to recurrent neural network language models, to adapt their predictions to local changes in the data distribution. These models only capture the local context, of up to a few…
Continual learning denotes machine learning methods which can adapt to new environments while retaining and reusing knowledge gained from past experiences. Such methods address two issues encountered by models in non-stationary…
This paper proposes a new sequential model learning architecture to solve partially observable Markov decision problems. Rather than compressing sequential information at every timestep as in conventional recurrent neural network-based…
The article provides a new approach to creating hierarchical structure of file system. First, it gives overview of the existing ways of storing files in current operating systems. Second, it describes the new way of building structures of a…
We address combinatorial problems that can be formulated as minimization of a partially separable function of discrete variables (energy minimization in graphical models, weighted constraint satisfaction, pseudo-Boolean optimization, 0-1…
The aim of the present paper is to investigate the half-spaces in the convexity structure of all quasiorders on a given set and to use them in an alternative approach to classical order dimension. The main result states that linear orders…