Related papers: How Do Transformers "Do" Physics? Investigating th…
Modern power systems are at risk of largely reducing the inertia of generation assets and prone to experience extreme dynamics. The consequence is that, during electromechanical transients triggered by large contingencies, transmission of…
How can Transformers model and learn enumerative geometry? What is a robust procedure for using Transformers in abductive knowledge discovery within a mathematician-machine collaboration? In this work, we introduce a Transformer-based…
Transformer becomes the state-of-the-art translation model, while it is not well studied how each intermediate component contributes to the model performance, which poses significant challenges for designing optimal architectures. In this…
Oscillators and rotators are among the most important physical systems. For centuries the only known rotating systems that actually reached the limits of the ideal situation of undamped periodical motion were the planets in their orbits.…
Ising Machines (IMs) are physical systems designed to find solutions to combinatorial optimization (CO) problems mapped onto the IM via the coupling strengths of its binary spins. Using the intrinsic dynamics and different annealing…
Hybrid oscillator architectures that combine feedback oscillators with self-sustained negative resistance oscillators have emerged as a promising platform for artificial neuron design. In this work, we introduce a modeling and analysis…
Singularities in macroscopic systems at discontinuous phase transitions are replaced in finite systems by sharp but continuous changes. Both the energy differences between metastable and stable phases and the energy barriers separating…
Time-dependent frequency harmonic oscillators (TDFHO's) are studied through the theory of Lie systems. We show that they are related to a certain kind of equations in the Lie group SL(2,R). Some integrability conditions appear as conditions…
Simulating quantum dynamics is one of the most important applications of quantum computers. Traditional approaches for quantum simulation involve preparing the full evolved state of the system and then measuring some physical quantity.…
A harmonic oscillator is an indefinite-frequency one if the parameter $\omega$ is replaced by an operator. An ensemble of $N$ such oscillators may be regarded as a toy model of a bosonic quantum field. All the possible frequencies…
To study the dynamical behaviour of the engineering and physical systems, we often need to capture their continuous behaviour, which is modeled using differential equations, and perform the frequency-domain analysis of these systems.…
We consider a discrete system of $n$ simple indistinguishable devices, called \emph{agents}, forming a \emph{connected} shape $S_I$ on a two-dimensional square grid. Agents are equipped with a linear-strength mechanism, called a \emph{line…
Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences. We introduce two techniques to improve the efficiency of…
A nonlinear two-dimensional system is studied by making use of both the Lagrangian and the Hamiltonian formalisms. The present model is obtained as a two-dimensional version of a one-dimensional oscillator previously studied at the…
Recurrent Neural Networks were, until recently, one of the best ways to capture the timely dependencies in sequences. However, with the introduction of the Transformer, it has been proven that an architecture with only attention-mechanisms…
The Transformer architecture has revolutionized the field of sequence modeling and underpins the recent breakthroughs in large language models (LLMs). However, a comprehensive mathematical theory that explains its structure and operations…
Transformers are a popular choice for classification tasks and as backbones for object detection tasks. However, their high latency brings challenges in their adaptation to lightweight object detection systems. We present an approximation…
Humans are able to seamlessly visually imitate others, by inferring their intentions and using past experience to achieve the same end goal. In other words, we can parse complex semantic knowledge from raw video and efficiently translate…
As transformer-based language models are trained on increasingly large datasets and with vast numbers of parameters, finding more efficient alternatives to the standard Transformer has become very valuable. While many efficient Transformers…
Recent works have shown that transformers can solve contextual reasoning tasks by internally executing computational graphs called circuits. Circuits often use attention to logically match information from subspaces of the representation,…