Related papers: On Transformations of Load-Store Maurer Instructio…
Information theory and the framework of information dynamics have been used to provide tools to characterise complex systems. In particular, we are interested in quantifying information storage, information modification and information…
Phase change memory has been developed into a mature technology capable of storing information in a fast and non-volatile way, with potential for neuromorphic computing applications. However, its future impact in electronics depends…
It is shown that the electronic conduction in silicon-on-insulator (SOI) layers exhibits a metallic regime which is very similar to that in high-mobility Si-metal oxide semiconductor structures (MOS). The peak in the electron mobility…
Transformers have become the go-to architecture for language and vision tasks, yet their theoretical properties, especially memorization capacity, remain elusive. This paper investigates the memorization abilities of multi-head attention…
The instruction learning paradigm -- where a model learns to perform new tasks from task descriptions alone -- has become popular in general-purpose model research. The capabilities of large transformer models as instruction learners,…
The development of neuromorphic systems based on memristive elements - resistors with memory - requires a fundamental understanding of their collective dynamics when organized in networks. Here, we study an experimentally inspired model of…
We present a comprehensive phenomenological model for the crossbar integrated metal-oxide continuous-state memristors. The model consists of static and dynamic equations, which are obtained by fitting a large amount of experimental data,…
Motor adaptation displays a structure-learning effect: adaptation to a new perturbation occurs more quickly when the subject has prior exposure to perturbations with related structure. Although this `learning-to-learn' effect is well…
Recently multimodal transformer models have gained popularity because their performance on language and vision tasks suggest they learn rich visual-linguistic representations. Focusing on zero-shot image retrieval tasks, we study three…
Although much progress has been made towards robust deep learning, a significant gap in robustness remains between real-world perturbations and more narrowly defined sets typically studied in adversarial defenses. In this paper, we aim to…
We perform experimental and numerical studies of a granular system under cyclic-compression to investigate reversibility and memory effects. We focus on the quasi-static forcing of dense systems, which is most relevant to a wide range of…
Many machine learning approaches are characterized by information constraints on how they interact with the training data. These include memory and sequential access constraints (e.g. fast first-order methods to solve stochastic…
We investigate machine learning approaches for optimizing real-time staffing decisions in semi-automated warehouse sortation systems. Operational decision-making can be supported at different levels of abstraction, with different…
In the interleaving model of concurrency, where events are totally ordered, linearizability is compositional: the composition of two linearizable objects is guaranteed to be linearizable. However, linearizability is not compositional when…
We study restrictions imposed by quantum mechanics on the process of matrix elements transfer. This problem is at the core of quantum measurements and state transfer. Given two systems $\A$ and $\B$ with initial density matrices $\lambda$…
We study the complexity of closure operators, with applications to machine learning and decision theory. In machine learning, closure operators emerge naturally in data classification and clustering. In decision theory, they can model…
We investigate the integration of human-like working memory constraints into the Transformer architecture and implement several cognitively inspired attention variants, including fixed-width windows based and temporal decay based attention…
We consider the computational power of silent transitions in one-way automata with storage. Specifically, we ask which storage mechanisms admit a transformation of a given automaton into one that accepts the same language and reads at least…
Constructing first principles models is a challenging task for nonlinear and complex systems such as a wastewater treatment unit. In recent years, data-driven models are widely used to overcome the complexity. However, they often suffer…
Disordered solids often change their elastic response as they slowly age. Using experiments and simulations, we study how aging disordered planar networks under an applied stress affects their nonlinear elastic response. We are able to…