Related papers: The First Principle of Big Memory Systems
Memory is fundamental to intelligence, enabling learning, reasoning, and adaptability across biological and artificial systems. While Transformer architectures excel at sequence modeling, they face critical limitations in long-range context…
Continual learning is considered a promising step towards next-generation Artificial Intelligence (AI), where deep neural networks (DNNs) make decisions by continuously learning a sequence of different tasks akin to human learning…
Memory is a complex phenomenon that involves several distinct mechanisms. These mechanisms operate at different spatial and temporal levels. This chapter focuses on the theoretical framework and the mathematical models that have been…
The semiconductor and IC industry is facing the issue of high energy consumption. In modern days computers and processing systems are designed based on the Turing machine and Von Neumann's architecture. This architecture mainly focused on…
An algorithm for first-principles electronic structure calculations having a computational cost which scales linearly with the system size is presented. Our method exploits the real-space localization of the density matrix, and in this…
Memory is often defined as the mental capacity of retaining information about facts, events, procedures and more generally about any type of previous experience. Memories are remembered as long as they influence our thoughts, feelings, and…
High capacity and scalable memory systems play a vital role in enabling our desktops, smartphones, and pervasive technologies like Internet of Things (IoT). Unfortunately, memory systems are becoming increasingly prone to faults. This is…
We introduce a world vision in which data is endowed with memory. In this data-centric systems paradigm, data items can be enabled to retain all or some of their previous values. We call this ability "remembrance" and posit that it empowers…
This article deals with dynamical systems depending on a slowly varying parameter. We present several physical examples illustrating memory effects, such as metastability and hysteresis, which frequently appear in these systems. A…
First passage time statistics in disordered systems exhibiting scale invariance are studied widely. In particular, long trapping times in energy or entropic traps are fat-tailed distributed, which slow the overall transport process. We…
Memory latency, bandwidth, capacity, and energy increasingly limit performance. In this paper, we reconsider proposed system architectures that consist of huge (many-terabyte to petabyte scale) memories shared among large numbers of CPUs.…
A cut-and-paste model which mimics a trial-and-error process of adaptation is introduced and solved. The model, which can be thought of as a diffusion process with memory, is characterized by two properties, efficiency and persistence. We…
Storage and retrieval of data in a computer memory plays a major role in system performance. Traditionally, computer memory organization is static - i.e., they do not change based on the application-specific characteristics in memory access…
Large-scale key-value storage systems sacrifice consistency in the interest of dependability (i.e., partition tolerance and availability), as well as performance (i.e., latency). Such systems provide eventual consistency,which---to this…
Persistent memory provides high-performance data persistence at main memory. Memory writes need to be performed in strict order to satisfy storage consistency requirements and enable correct recovery from system crashes. Unfortunately,…
As the final stage of recommender systems, re-ranking presents ordered item lists to users that best match their interests. It plays such a critical role and has become a trending research topic with much attention from both academia and…
We consider first and second order consensus algorithms in networks with stochastic disturbances. We quantify the deviation from consensus using the notion of network coherence, which can be expressed as an $H_2$ norm of the stochastic…
Over the past two decades, the storage capacity and access bandwidth of main memory have improved tremendously, by 128x and 20x, respectively. These improvements are mainly due to the continuous technology scaling of DRAM (dynamic…
Neural networks (NN)-based learning algorithms are strongly affected by the choices of initialization and data distribution. Different optimization strategies have been proposed for improving the learning trajectory and finding a better…
The semantics of HPC storage systems are defined by the consistency models to which they abide. Storage consistency models have been less studied than their counterparts in memory systems, with the exception of the POSIX standard and its…