Related papers: How Do Transformers "Do" Physics? Investigating th…

Transformer-based models have recently become wildly successful across a diverse set of domains. At the same time, recent work has shown empirically and theoretically that Transformers are inherently limited. Specifically, they argue that…

We study the unitary time evolution of a simple quantum Hamiltonian describing two harmonic oscillators coupled via a three-level system. The latter acts as an engine transferring energy from one oscillator to the other and is driven in a…

Determining whether a program terminates is a core challenge in program analysis with direct implications for correctness, verification, and security. We investigate whether transformer architectures can recognise termination patterns…

Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…

It is shown in first order perturbation theory that anharmonic oscillators in non-commutative space behave smoothly in the commutative limit just as harmonic oscillators do. The non-commutativity provides a method for converting a problem…

Transformer architectures excel at sequential modeling yet remain fundamentally limited by correlational learning - they capture spurious associations induced by latent confounders rather than invariant causal mechanisms. We identify this…

Building oscillator based computing systems with emerging nano-device technologies has become a promising solution for unconventional computing tasks like computer vision and pattern recognition. However, simulation and analysis of these…

We explore the capability of transformers to address endogeneity in in-context linear regression. Our main finding is that transformers inherently possess a mechanism to handle endogeneity effectively using instrumental variables (IV).…

Supersymmetry plays a main role in all current thinking about superstring theory. Indeed, many remarkable properties of string theory have been explained using supersymmetry as a tool. So far, there has been no unbroken supersymmetry…

The paper reports on a study of a harmonic oscillator (ho) in the twisted Moyal space, in a well defined matrix basis, generated by the vector fields…

We present a methodology for training foundational transformer models capable of processing collider data with diverse kinematic signatures. Our universal foundation model is designed for simultaneous analysis of all processes involving…

We have developed a physics-informed transformer model to suggest design parameters in wavelength-filtering ring resonator, that suit a given pair of resonant wavelengths with <6 nm errors. The model provides a versatile method for rapid…

With the attention mechanism, transformers achieve significant empirical successes. Despite the intuitive understanding that transformers perform relational inference over long sequences to produce desirable representations, we lack a…

There is evidence that transformers offer state-of-the-art recognition performance on tasks involving overhead imagery (e.g., satellite imagery). However, it is difficult to make unbiased empirical comparisons between competing deep…

We report on the possibilities of using the method of normal fundamental systems for solving some problems of oscillation theory. Large elastic dynamical systems with continuous and discrete parameters are considered, which have many…

In this paper we review recent work on novel computing paradigms using coupled oscillatory dynamical systems. We explore systems of relaxation oscillators based on linear state transitioning devices, which switch between two discrete states…

What is the computational model behind a Transformer? Where recurrent neural networks have direct parallels in finite state machines, allowing clear discussion and thought around architecture variants or trained models, Transformers have no…

This paper investigates the use of transformers to approximate the mean-field dynamics of interacting particle systems exhibiting collective behavior. Such systems are fundamental in modeling phenomena across physics, biology, and…

Transformers have recently been utilized to perform object detection and tracking in the context of autonomous driving. One unique characteristic of these models is that attention weights are computed in each forward pass, giving insights…

Shell structure in the single particle spectrum of deformed harmonic oscillator potentials when a term proportional to $(\vec L)^2$ is added is analyzed for a large particle number. A scaling law which gives a dividing line between regular…