Related papers: How Do Transformers "Do" Physics? Investigating th…
The harmonic oscillator is the paragon of physical models; conceptually and computationally simple, yet rich enough to teach us about physics on scales that span classical mechanics to quantum field theory. This multifaceted nature extends…
The periodically driven harmonic oscillator with damping is one of the most elementary and trusted models in physics and normally applied in its steady state, disregarding specific initial conditions and associated transients. For example,…
The Fourier series method is used to solve the homogeneous equation governing the motion of the harmonic oscillator. It is shown that the general solution to the problem can be found in a surprisingly simple way for the case of the simple…
The transformer is a neural network component that can be used to learn useful representations of sequences or sets of data-points. The transformer has driven recent advances in natural language processing, computer vision, and…
A harmonic oscillator with time-dependent mass $m(t)$ and a time-dependent (squared) frequency $\omega^2(t)$ occurs in the modelling of several physical systems. It is generally believed that systems, with $m(t)>0$ and $\omega^2(t)>0$…
Many real-world problems can be naturally described by mathematical formulas. The task of finding formulas from a set of observed inputs and outputs is called symbolic regression. Recently, neural networks have been applied to symbolic…
Algorithmic reasoning requires capabilities which are most naturally understood through recurrent models of computation, like the Turing machine. However, Transformer models, while lacking recurrence, are able to perform such reasoning…
The chaotic properties of simple two-dimensional rotation-translation models are explored and simulated. The models are given in difference equation forms, while the corresponding differential equations systems are studied and the resulting…
Understanding how information propagates through Transformer models is a key challenge for interpretability. In this work, we study the effects of minimal token perturbations on the embedding space. In our experiments, we analyze the…
Foundation models exhibit significant capabilities in decision-making and logical deductions. Nonetheless, a continuing discourse persists regarding their genuine understanding of the world as opposed to mere stochastic mimicry. This paper…
Transformers are widely used to extract semantic meanings from input tokens, yet they usually operate as black-box models. In this paper, we present a simple yet informative decomposition of hidden states (or embeddings) of trained…
The purpose of this article is the study of the symmetries in a circular and linear harmonic oscillator chains system, and consequently use them as a means to find the eigenvalues of these configurations. Furthermore, a hidden…
Transformers excel at time series modelling through attention mechanisms that capture long-term temporal patterns. However, they assume uniform time intervals and therefore struggle with irregular time series. Neural Ordinary Differential…
Transformers have become the architecture of choice for learning long-range dependencies, yet their adoption in hyperspectral imaging (HSI) is still emerging. We reviewed more than 300 papers published up to 2025 and present the first…
Neutrino oscillations encode fundamental information about neutrino masses and mixing parameters, offering a unique window into physics beyond the Standard Model. Estimating these parameters from oscillation probability maps is, however,…
Precision tests of the Standard Model and searches for beyond the Standard Model physics often require nuclear structure input. There has been a tremendous progress in the development of nuclear ab initio techniques capable of providing…
Transformers generate valid and diverse chemical structures, but little is known about the mechanisms that enable these models to capture the rules of molecular representation. We present a mechanistic analysis of autoregressive…
Other than scattering problems where perturbation theory is applicable, there are basically two ways to solve problems in physics. One is to reduce the problem to harmonic oscillators, and the other is to formulate the problem in terms of…
The Schwinger's representation of angular momentum(AM) relates two important fundamental models, that of AM and that of harmonic oscillator(HO). However, the representation offers only the relations of operators but not states. Here, by…
Mathematical models are increasingly being used to understand complex biochemical systems, to analyze experimental data and make predictions about unobserved quantities. However, we rarely know how robust our conclusions are with respect to…