Related papers: On Transformations of Load-Store Maurer Instructio…
Deep neural state-space models (SSMs) provide a powerful tool for modeling dynamical systems solely using operational data. Typically, neural SSMs are trained using data collected from the actual system under consideration, despite the…
Non-Markovian quantum processes exhibit different memory effects when measured in different ways; an unambiguous characterization of memory length requires accounting for the sequence of instruments applied to probe the system dynamics.…
We conduct a systematic study of the approximation properties of Transformer for sequence modeling with long, sparse and complicated memory. We investigate the mechanisms through which different components of Transformer, such as the…
A load sharing system has several components and the failure of one component can affect the lifetime of the surviving components. Since component failure does not equate to system failure for different system designs, the analysis of the…
Quantum memory is a key element for quantum repeaters and linear optical quantum computers. In addition to memory, repeaters and computers also require manipulating quantum states by means of unitary transformations, which is generally…
To enhance the efficiency of capacity markets, many electricity markets in the U.S. are adopting or planning to implement marginal capacity accreditation reforms. This paper provides new insights into energy storage capacity accreditation…
Non-equilibrium systems display memory, a dependence not merely on their present environment but on previously applied fields. Multistable systems such as spin glasses, martensites and granular matter have exponentially many microstates…
Using conditional measurement on a beam splitter, we study the transformation of the quantum state of the signal mode within the concept of two-port non-unitary transformation. Allowing for arbitrary quantum states of both the input…
In a sequential decision-making problem, the information structure is the description of how events in the system occurring at different points in time affect each other. Classical models of reinforcement learning (e.g., MDPs, POMDPs)…
Transformers have demonstrated effectiveness in in-context solving data-fitting problems from various (latent) models, as reported by Garg et al. However, the absence of an inherent iterative structure in the transformer architecture…
Multiprocess systems, including grid systems, multiprocessors and multicore computers, incorporate a variety of specialized hardware and software mechanisms, which speed computation, but result in complex memory behavior. As a consequence,…
Recent large language models have been trained on vast datasets, but also often on repeated data, either intentionally for the purpose of upweighting higher quality data, or unintentionally because data deduplication is not perfect and the…
In this paper, we view a policy or plan as a transition system over a space of information states that reflect a robot's or other observer's perspective based on limited sensing, memory, computation, and actuation. Regardless of whether…
We discuss memory effects in the conductance of hopping insulators due to slow rearrangements of many-electron clusters leading to formation of polarons close to the electron hopping sites. An abrupt change in the gate voltage and…
The military environment generates a large amount of data of great importance, which makes necessary the use of machine learning for its processing. Its ability to learn and predict possible scenarios by analyzing the huge volume of…
We present Shapechanger, a library for transfer reinforcement learning specifically designed for robotic tasks. We consider three types of knowledge transfer---from simulation to simulation, from simulation to real, and from real to…
Can we change the average state of a resistor by simply applying white noise? We show that the answer to this question is positive if the resistor has memory of its past dynamics (a memristive system). We also prove that, if the memory…
This work is concerned with the stability of regime-switching processes under the perturbation of the transition rate matrices. From the viewpoint of application, two kinds of perturbations are studied: the size of the transition rate…
Long Short-Term Memory (LSTM) is one of the most widely used recurrent structures in sequence modeling. It aims to use gates to control information flow (e.g., whether to skip some information or not) in the recurrent computations, although…
We study how a one-layer attention-only transformer develops relevant structures while learning to sort lists of numbers. At the end of training, the model organizes its attention heads in two main modes that we refer to as…